Re: [PATCH v4 0/4] Introduce Per-CPU Work helpers (was QPW)

[Leonardo Bras]

On Wed, May 20, 2026 at 03:09:03PM +0200, Sebastian Andrzej Siewior wrote:
> On 2026-05-18 22:27:46 [-0300], Leonardo Bras wrote:
> > The problem:
> > Some places in the kernel implement a parallel programming strategy
> > consisting on local_locks() for most of the work, and some rare remote
> > operations are scheduled on target cpu. This keeps cache bouncing low since
> > cacheline tends to be mostly local, and avoids the cost of locks in non-RT
> > kernels, even though the very few remote operations will be expensive due
> > to scheduling overhead.
> > 
> > On the other hand, for RT workloads this can represent a problem: getting
> > an important workload scheduled out to deal with remote requests is
> > sure to introduce unexpected deadline misses.
> > 
> > The idea:
> > Currently with PREEMPT_RT=y, local_locks() become per-cpu spinlocks.

Hi Sebastian, thank you for reviewing!
(Sorry for the delay)

> It does not become a _spin_lock because it does not spin. It sleeps.

Right, it's a per-cpu mutex. 
My point is that it's a full lock, and we could use it instead of doing the 
whole scheduling thing, since we are already paying the 'atomic overhead' 
to get the lock here.

> 
> > In this case, instead of scheduling work on a remote cpu, it should
> > be safe to grab that remote cpu's per-cpu spinlock and run the required
> > work locally. That major cost, which is un/locking in every local function,
> > already happens in PREEMPT_RT.
> 
> We did have this before but only in the RT tree. It was a bit messy from
> the naming because it started with local_ but then it was a remote CPU.

Had the same naming issue here. This idea was initially a expansion to 
local_lock() mechanism, about the same way you were planning in the past.

> The main issue was the different code path which led to a few deadlocks
> back then.
> By the time local_lock_t went upstream, the cross-CPU locking was
> removed. As far as I remember, the cross-CPU user which did schedule
> work on a remote CPU and annoyed NOHZ folks were replaced.

I understand this could be a big issue if used in a generic way.

What I am proposing here a mechanism that standardizes those 
local_lock()+IPI strategies based on how they are done today, so we are 
only explected to get 'remote-cpu' pwlocks in the 'IPI replacement' 
operations.

The idea is pwlock_local* in every local function, and pwlock*(,cpu) in 
operations that can be remote. 

Maybe being used in a more constrained way, it has less chance of being an 
issue. Also, the whole idea is to improve CPU isolation numbers by 
reducing IPIs, so maybe NOHZ people will be happier with that :)

Thanks again!
Leo