Re: [PATCH 0/3] sparc64: stale TLB/TSB entries can livelock a thread

[Magnus Lindholm]

Hi Stian,

On Sat, Oct 3, 2026 at 2:56 PM Stian Halseth <stian@xxxxxx> wrote:
>
> On a SPARC T7-1 (M7, 256 strands) running up to 256 compiler processes
> in parallel, a thread now and then stops dead: runnable, fixed PC,
> system time climbing, no kernel messages.  One was caught retrying the
> same store for 25 minutes.
>
> Two causes:
>
>  - flush_tlb_fix_spurious_fault() is a no-op on sparc64 (it falls back
>    to flush_tlb_page(), which is empty here), so a write that faults on
>    a stale read-only TLB entry faults again forever.  Same for the PMD
>    hook.
>
>  - tsb_grow() publishes the new TSB before the other CPUs have switched
>    to it, and all flushes go to the new one.  Stale entries survive in
>    the old TSB and are reloaded into the TLB: read-only ones feed the
>    loop above, writable ones for unmapped pages point at freed memory.
>
> A probe in the fault path on the T7-1 showed 128 of 868 spurious faults
> during one toolchain build and test run on a CPU still using an old TSB
> that held the read-only entry, 31 of them repeating on the same thread
> and page.  A SPARC T4-1 (64 strands) shows the same: 143 of 2780.  With
> the series the T7-1 run shows 757 spurious faults, none on an old TSB
> and no repeats, and the build's system time is back from 29 min to 7.
>
> Patch 1 avoids a double flush once the hook does real work, patch 2
> defines the PTE and PMD hooks, patch 3 keeps the old TSB flushable
> until smp_tsb_sync() has returned.
>
> Tested on a T7-1 (sun4v): five consecutive full Go test suite runs
> (make.bash + dist test), once with THP off and once with THP enabled,
> no stalls; a T4-1 runs it as its daily kernel.
> On sun4u (V240, UltraSPARC IIIi) it boots and survives TSB-grow,
> copy-on-write, hugetlb and fork stress with no stale reads.
> Build-tested with CONFIG_SMP=n and without THP/hugetlb.
>
> Tracking issue with the probe patch and the logs:
> https://github.com/sparclinux/issues/issues/108
>
> Stian Halseth (3):
>   sparc64: provide ptep_set_access_flags()
>   sparc64: flush the TLB on a spurious write fault
>   sparc64: keep flushing the old TSB while tsb_grow() switches CPUs
>
>  arch/sparc/include/asm/mmu_64.h      |   5 ++
>  arch/sparc/include/asm/pgtable_64.h  |   3 +
>  arch/sparc/include/asm/tlbflush_64.h |  10 ++++
>  arch/sparc/mm/tlb.c                  |  35 ++++++++++++++
>  arch/sparc/mm/tsb.c                  |  80 ++++++++++++++++++++++++++++++-
>  5 files changed, 127 insertions(+), 6 deletions(-)
>

Tested on an M3000 (SPARC64 VII+, eight CPUs) and an Ultra45
(UltraSPARC IIIi, UP). Both booted and passed memory-growth,
COW and remap/protection tests without observed regressions.
The Ultra45 also passed HugeTLB mapping tests.

Counters on the M3000 confirmed execution of the spurious-fault
hook and both old-TSB flush paths. I did not reproduce the original
livelock before patching, and THP was not tested.

The M3000 tests included a separate integration adjustment for
my unmerged Fujitsu support: physical-address conversion in the
four new old-TSB flush paths. Your three patches were unchanged.

So for this series:

Tested-by: Magnus Lindholm <linmag7@xxxxxxxxx>