Re: [PATCH] sparc32: sun4m: do not clear IPIs that were not handled

[Magnus Lindholm]

Hi Ramin,

On Sun, Oct 18, 2026 at 11:53 AM Ramin Moussavi
<ramin.moussavi@xxxxxxxxx> wrote:
>
> maybe_smp4m_msg reads the per-cpu interrupt pending register and then
> writes 0x70000000 to the clear register, acknowledging all three soft
> interrupt levels used for IPIs (12, 13 and 14) whether or not they were
> pending in the value it read. An IPI that another CPU raises between the
> read and the write is acknowledged without ever being handled.
>
> For IRQ_IPI_SINGLE this is not just a delay. __smp_call_single_queue()
> only sends the IPI when the target's call_single_queue was empty, so
> once one IPI is lost no further IPI is sent to that CPU, and every
> remote wakeup queued there waits until the CPU next goes idle and
> flushes the queue. Under CPU-bound load that does not happen: woken
> tasks show up as R in /proc, with a vruntime below that of the tasks
> that keep running, and still do not run until the load ends.
>
> Seen on a 4-CPU SPARCstation 20 under qemu with eight threads doing
> syscalls and window flushes next to threads that wake each other
> through condition variables and pipes: 7 of 8 runs had tasks starving
> for as long as the load lasted (up to 25 minutes), 0 of 4 runs with a
> single CPU. With this change 0 of 8 runs starved.
>
> Acknowledge only the levels that were seen. A level raised after the
> read stays pending and traps again once PIL allows.
>
> Fixes: ecbc42b70acb ("sparc32, sun4m: Implemented SMP IPIs support for SUN4M machines")
> Cc: stable@xxxxxxxxxxxxxxx
> Assisted-by: Claude:claude-opus-5-5
> Signed-off-by: Ramin Moussavi <ramin.moussavi@xxxxxxxxx>
> ---
>  arch/sparc/kernel/entry.S | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/arch/sparc/kernel/entry.S b/arch/sparc/kernel/entry.S
> index ea51ef52c952..0bc30436f028 100644
> --- a/arch/sparc/kernel/entry.S
> +++ b/arch/sparc/kernel/entry.S
> @@ -275,11 +275,11 @@ maybe_smp4m_msg:
>         sethi   %hi(0x70000000), %o2    ! Check all soft-IRQs
>         ld      [%o5 + %o3], %o1
>         ld      [%o1 + 0x00], %o3       ! sun4m_irq_percpu[cpu]->pending
> -       andcc   %o3, %o2, %g0
> +       andcc   %o3, %o2, %o4
>         be,a    smp4m_ticker
>          cmp    %l7, 14
> -       /* Soft-IRQ IPI */
> -       st      %o2, [%o1 + 0x04]       ! sun4m_irq_percpu[cpu]->clear=0x70000000
> +       /* Soft-IRQ IPI: clear only the ones we saw, a later one must stay */
> +       st      %o4, [%o1 + 0x04]       ! sun4m_irq_percpu[cpu]->clear
>         WRITE_PAUSE
>         ld      [%o1 + 0x00], %g0       ! sun4m_irq_percpu[cpu]->pending
>         WRITE_PAUSE
> --
> 2.53.0
>

I tested this on a dual-CPU HyperSPARC (RT626) SPARCstation 20 with 512 MB
RAM, using a 32-bit SMP kernel with your patch, Imre's PIL fix and my
HyperSPARC cache changes (not yet published, work in progress).

After a 60-second sanity check, I ran an independent 30-minute stress test:
eight pinned threads issuing syscalls and flushing register windows,
alongside two condition-variable pairs and two pipe pairs waking across
CPUs. All 16 workers kept making progress under load and the test passed.
There were no new core dumps, kernel warnings or RCU stalls.

The same workload also passed a 60-second run with four CPUs under QEMU.

I'm working on HyperSPARC cache-coherency fixes and have also encountered
SMP livelocks where interrupts remain responsive but tasks stop making
progress. I haven't established whether all of these are cache-related, so
this lost-IPI race is particularly interesting.

Tested-by: Magnus Lindholm <linmag7@xxxxxxxxx>

Thanks,
Magnus