Re: [REGRESSION] osnoise: "eventpoll: Replace rwlock with spinlock" causes ~50µs noise spikes on isolated PREEMPT_RT cores

[Sebastian Andrzej Siewior]

On 2026-03-26 16:00:57 [+0200], Ionut Nechita (Wind River) wrote:
> Summary across all isolated cores (32 CPUs):
> 
>                           With spinlock       With rwlock (reverted)
>   MAX noise (ns):         44,343 - 51,869     0 - 10
>   IRQ count/sample:       ~6,650 - 6,870      3 - 7
>   Thread noise/sample:    ~5,700 - 5,940      0 - 1
>   CPU availability:       94.5% - 95.3%       ~100%

is there some load or just idle with osnoise?

> My understanding of the root cause: the original rwlock allowed
> ep_poll_callback() (producer side, running from IRQ context on any CPU)
> to use read_lock, which does not cause cross-CPU contention on isolated
> cores when no local epoll activity exists. With the spinlock conversion,
> on PREEMPT_RT spinlock_t becomes an rt_mutex. This means that even if
> the isolated core is not involved in any epoll activity, the lock's
> cacheline bouncing and potential PI-boosted wakeups from housekeeping
> CPUs can inject noise into the isolated cores via IPI or cache
> invalidation traffic.

With the read_lock() you can acquire the lock with multiple readers.
Each read will increment the "reader counter" so there is cache line
activity. If a isolated CPU does not participate, it does not
participate. With the change to spinlock_t there can be only one user at
a time. So the other have to wait and again, isolated core which don't
participate are not affected.

> The commit message acknowledges the throughput regression but argues
> real workloads won't notice. However, for RT/latency-sensitive
> deployments with CPU isolation, the impact is severe and measurable
> even with zero local epoll usage.
> 
> I believe this needs either:
>   a) A revert of the backport for stable RT trees, or

I highly doubt since it affected RT loads.

>   b) A fix that avoids the spinlock contention path for isolated CPUs
> 
> I can provide the full osnoise trace data if needed.

So the question is why are the isolated core affected if they don't
participate is epoll.

> Tested on:
>   Linux system-0 6.12.78-vanilla-{0,1} SMP PREEMPT_RT x86_64
>   Linux system-0 6.12.57-vanilla-{0,1} SMP PREEMPT_RT x86_64
> 
> Thanks,
> Ionut.

Sebastian