Re: [PATCH v9 4/7] arm64: smp: Assign and setup the debug IPI

From: Mark Rutland
Date: Mon Aug 07 2023 - 06:17:57 EST


On Thu, Jun 01, 2023 at 02:31:48PM -0700, Douglas Anderson wrote:
> From: Sumit Garg <sumit.garg@xxxxxxxxxx>
>
> All current arm64 interrupt controllers have at least 8
> IPIs. Currently we are only using 7 of them on arm64. Let's use the
> 8th one as a debug IPI. This uses the new "debug IPI" infrastructure
> which will try to allocate this IPI as an NMI/pseudo NMI if possible.
>
> Signed-off-by: Sumit Garg <sumit.garg@xxxxxxxxxx>
> Signed-off-by: Douglas Anderson <dianders@xxxxxxxxxxxx>

I think with my suggestion on the prior patch, we don't need the additional
logic here.

> ---
> I could imagine that people object to using up the last free IPI on
> interrupt controllers with only 8 IPIs. However, it shouldn't be a big
> deal. If we later need an extra IPI, it shouldn't be too hard to
> combine some of the existing ones. Presumably we could just get rid of
> the "crash stop" IPI and have the normal "stop" IPI do the crash if
> "waiting_for_crash_ipi" is non-zero

TBH, I'd love to unify the logic for IPI_CPU_STOP and IPI_CPU_CRASH_STOP as
they have a bunch of pointless divergence.

We could also remove IPI_WAKEUP and replace that with something else (e.g.
IPI_CALL_FUNC with an empty function) in order to free up an IPI.

I'd *also* prefer to have separate IPI_CPU_BACKTRACE and IPI_CPU_DEBUG as the
backtrace logic is used for a bunch of things other than KGDB (e.g. RCU stalls,
panic), and that would clearly separate the logic for the two cases.

Thanks,
Mark.

>
> Changes in v9:
> - Add a warning if we don't have enough IPIs for the NMI IPI
> - Renamed "NMI IPI" to "debug IPI" since it might not be backed by NMI.
> - Update commit description
>
> Changes in v8:
> - debug_ipi_setup() and debug_ipi_teardown() no longer take cpu param
>
> arch/arm64/kernel/smp.c | 11 +++++++++++
> 1 file changed, 11 insertions(+)
>
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index edd63894d61e..db019b49d3bd 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -53,6 +53,8 @@
>
> #include <trace/events/ipi.h>
>
> +#include "ipi_debug.h"
> +
> DEFINE_PER_CPU_READ_MOSTLY(int, cpu_number);
> EXPORT_PER_CPU_SYMBOL(cpu_number);
>
> @@ -935,6 +937,8 @@ static void ipi_setup(int cpu)
>
> for (i = 0; i < nr_ipi; i++)
> enable_percpu_irq(ipi_irq_base + i, 0);
> +
> + debug_ipi_setup();
> }
>
> #ifdef CONFIG_HOTPLUG_CPU
> @@ -947,6 +951,8 @@ static void ipi_teardown(int cpu)
>
> for (i = 0; i < nr_ipi; i++)
> disable_percpu_irq(ipi_irq_base + i);
> +
> + debug_ipi_teardown();
> }
> #endif
>
> @@ -968,6 +974,11 @@ void __init set_smp_ipi_range(int ipi_base, int n)
> irq_set_status_flags(ipi_base + i, IRQ_HIDDEN);
> }
>
> + if (n > nr_ipi)
> + set_smp_debug_ipi(ipi_base + nr_ipi);
> + else
> + WARN(1, "Not enough IPIs for NMI IPI\n");
> +
> ipi_irq_base = ipi_base;
>
> /* Setup the boot CPU immediately */
> --
> 2.41.0.rc2.161.g9c6817b8e7-goog
>