Re: [PATCH] irqchip/gic-v3-its: Select housekeeping CPUs preferentially for managed IRQs

[Marc Zyngier]

+ John Garry, as he was reporting issues around the same piece of code[1]

On Mon, 24 Jan 2022 07:34:40 +0000,
Xiongfeng Wang <wangxiongfeng2@xxxxxxxxxx> wrote:
> 
> When using kernel parameter 'isolcpus=managed_irq,xxxx' to bind the
> managed IRQs to housekeeping CPUs, the effective_affinity sometimes
> still contains the non-housekeeping CPUs.
> 
> irq_do_set_affinity() passes the housekeeping cpumask to
> chip->irq_set_affinity(), but ITS driver select CPU according to
> irq_common_data->affinity. While 'irq_common_data->affinity' is updated
> after chip->irq_set_affinity() is called in irq_do_set_affinity(). Also
> 'irq_common_data->affinity' may contains non-housekeeping CPUs. I found
> the below link explaining the reason.
>   https://www.mail-archive.com/linux-kernel@xxxxxxxxxxxxxxx/msg2267032.html
> 
> To modify CPU selecting logic to prefer housekeeping CPUs, select CPU
> from the input cpumask parameter first. If none of it is online, then
> select CPU from 'irq_common_data->affinity'.
> 
> Signed-off-by: Xiongfeng Wang <wangxiongfeng2@xxxxxxxxxx>
> ---
>  drivers/irqchip/irq-gic-v3-its.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
> index d25b7a864bbb..17c15d3b2784 100644
> --- a/drivers/irqchip/irq-gic-v3-its.c
> +++ b/drivers/irqchip/irq-gic-v3-its.c
> @@ -1624,7 +1624,10 @@ static int its_select_cpu(struct irq_data *d,
>  
>  		cpu = cpumask_pick_least_loaded(d, tmpmask);
>  	} else {
> -		cpumask_and(tmpmask, irq_data_get_affinity_mask(d), cpu_online_mask);
> +		cpumask_and(tmpmask, aff_mask, cpu_online_mask);
> +		if (cpumask_empty(tmpmask))
> +			cpumask_and(tmpmask, irq_data_get_affinity_mask(d),
> +				    cpu_online_mask);

I think that the online_cpu_mask logical and is a bit wrong. A managed
interrupt should be able to target an offline CPU:

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index eb0882d15366..0cea46bdaf99 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -1620,7 +1620,7 @@ static int its_select_cpu(struct irq_data *d,
 
 		cpu = cpumask_pick_least_loaded(d, tmpmask);
 	} else {
-		cpumask_and(tmpmask, irq_data_get_affinity_mask(d), cpu_online_mask);
+		cpumask_copy(tmpmask, aff_mask);
 
 		/* If we cannot cross sockets, limit the search to that node */
 		if ((its_dev->its->flags & ITS_FLAGS_WORKAROUND_CAVIUM_23144) &&

We still have an issue when the system hasn't booted with all its
CPUs, as the corresponding collections aren't initialised and we
end-up in a rather bad place.

	M.

[1] https://lore.kernel.org/r/78615d08-1764-c895-f3b7-bfddfbcbdfb9@xxxxxxxxxx

-- 
Without deviation from the norm, progress is not possible.