Re: [PATCH v2] mm/slab: don't use kfree_rcu sheaves on PREEMPT_RT in kvfree_call_rcu()

[Sebastian Andrzej Siewior]

On 2026-08-31 13:06:32 [+0700], ThangNN99 wrote:
> syzbot reports a possible circular locking dependency between
> &p->pi_lock and the per-CPU kfree_rcu sheaf lock (_T->lock) on
> PREEMPT_RT:
> 
>   __balance_push_cpu_stop()               [holds p->pi_lock, raw]
>     select_fallback_rq()
>       cpuset_cpus_allowed_fallback()
>         set_cpus_allowed_force()
>           kfree_rcu(ac.user_mask)
>             kvfree_call_rcu()
>               kfree_rcu_sheaf()
>                 __kfree_rcu_sheaf()
>                   local_trylock(&s->cpu_sheaves->lock)   <- _T->lock
> 
> set_cpus_allowed_force() uses kfree_rcu() instead of kfree() here
> specifically because it can be called with p->pi_lock (a raw
> spinlock) held, and plain kfree() may sleep under PREEMPT_RT.

A raw_spinlock_t. Please don't invent new things.

…
> diff --git a/mm/slab_common.c b/mm/slab_common.c
> index b19ba1b31484..3de1eabe6c77 100644
> --- a/mm/slab_common.c
> +++ b/mm/slab_common.c
> @@ -1667,15 +1667,8 @@ static bool kfree_rcu_sheaf(void *obj)
>  {
>  	struct kmem_cache *s;
>  	struct slab *slab;
> -	unsigned int free_flags = SLAB_FREE_DEFAULT;
> -
> -	/*
> -	 * It is not safe to spin on PREEMPT_RT because the kernel might be
> -	 * holding a raw spinlock and slab acquires sleeping locks.
> -	 */
> -	if (IS_ENABLED(CONFIG_PREEMPT_RT))
> -		free_flags = SLAB_FREE_NOLOCK;
>  
> +	/* Callers on PREEMPT_RT never reach here, see kvfree_call_rcu(). */
>  	if (is_vmalloc_addr(obj))
>  		return false;
>  
> @@ -1685,7 +1678,7 @@ static bool kfree_rcu_sheaf(void *obj)
>  
>  	s = slab->slab_cache;
>  	if (likely(!IS_ENABLED(CONFIG_NUMA) || slab_nid(slab) == numa_mem_id()))
> -		return __kfree_rcu_sheaf(s, obj, free_flags);
> +		return __kfree_rcu_sheaf(s, obj, SLAB_FREE_DEFAULT);
>  
>  	return false;
>  }
> @@ -2034,7 +2027,14 @@ void kvfree_call_rcu(struct kvfree_rcu_head *head, void *ptr)
>  	if (!head)
>  		might_sleep();
>  
> -	if (kfree_rcu_sheaf(ptr))
> +	/*
> +	 * Callers may hold a raw spinlock here on PREEMPT_RT (e.g.
> +	 * set_cpus_allowed_force() with p->pi_lock held), and the sheaf/barn

No. All callers of set_cpus_allowed_force() hold task_struct::pi_lock.

> +	 * locks are also taken as blocking locks elsewhere, so trying them
> +	 * here creates a lockdep-visible ordering conflict. Skip sheaves on
> +	 * PREEMPT_RT.
> +	 */
> +	if (!IS_ENABLED(CONFIG_PREEMPT_RT) && kfree_rcu_sheaf(ptr))

Given that this can hold the pi_lock I am not a big of this trylock
underneath. So skipping it is my favorite.

>  		return;
>  
>  	// Queue the object but don't yet schedule the batch.
> diff --git a/mm/slub.c b/mm/slub.c
> index f9b56cb439e7..1e8bad7a018e 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -6088,10 +6088,10 @@ static void rcu_free_sheaf(struct rcu_head *head)
>  /*
>   * kvfree_call_rcu() can be called while holding a raw_spinlock_t. Since
>   * __kfree_rcu_sheaf() may acquire a spinlock_t (sleeping lock on PREEMPT_RT),
> - * this would violate lock nesting rules. Therefore, kvfree_call_rcu() avoids
> - * this problem by passing SLAB_FREE_NOLOCK on PREEMPT_RT.
> + * this would violate lock nesting rules. kvfree_call_rcu() avoids this by
> + * bypassing the sheaves layer on PREEMPT_RT.
>   *
> - * However, lockdep still complains that it is invalid to acquire spinlock_t
> + * lockdep still complains that it is invalid to acquire spinlock_t
>   * while holding raw_spinlock_t, even on !PREEMPT_RT where spinlock_t is a
>   * spinning lock. Tell lockdep that acquiring spinlock_t is valid here
>   * by temporarily raising the wait-type to LD_WAIT_CONFIG. Skip the lockdep map

Sebastian