Re: [PATCH RFC v2] mm, slab: add an optimistic __slab_try_return_freelist()

[Harry Yoo (Oracle)]

On Mon, Apr 27, 2026 at 11:42:57AM +0200, Vlastimil Babka (SUSE) wrote:
> When we end up returning extraneous objects during refill to a slab
> where we just did a get_freelist_nofreeze(), it is likely no other CPU
> has freed objects to it meanwhile. We can then reattach the remainder of
> the freelist without having to walk the (potentially cache cold)
> freelist for finding its tail to connect slab->freelist to it.
> 
> Add a __slab_try_return_freelist() function that does that. As suggested
> by Hao Li, it doesn't need to also return the slab to the partial list,
> because there's code in __refill_objects_node() that already does that
> for any slabs where we don't detach the freelist in the first place.
> 
> Reviewed-by: Hao Li <hao.li@xxxxxxxxx>
> Signed-off-by: Vlastimil Babka (SUSE) <vbabka@xxxxxxxxxx>
> ---
> Optimizes the current refill leftover handling in a way that should have
> no downsides, so we have a better baseline for any further changes (e.g.
> spilling or caching the leftover) that involve some tradeoffs.
> 
> Git version here:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=b4/refill-optimistic-return
> 
> It's based on slab/for-7.2/perf with "mm/slub: defer freelist
> construction until after bulk allocation from a new slab".
> ---
> Changes in v2:
> - rebase to slab/for-7.2/perf, drop RFC
> - simplify to reuse the existing reattaching to partial list (Hao Li)
> - Add R-b from Hao Li, thanks!
> - drop the stat items - they serverd to verify the optimistic path was
>   succeeding, but are too detailed for mainline
> - Link to v1: https://patch.msgid.link/20260421-b4-refill-optimistic-return-v1-1-24f0bfc1acff@xxxxxxxxxx
> ---

Looks good to me,
Reviewed-by: Harry Yoo (Oracle) <harry@xxxxxxxxxx>

with a few typo fixes below.

> @@ -5502,6 +5504,34 @@ static noinline void free_to_partial_list(
>  	}
>  }
>  
> +/*
> + * Try returning (remainder of) the freelist that we just detached from the
> + * slab.  Optimistically assume the slab is still full, so we don't need to find
> + * the tail of the detached freelist.
> + *
> + * Fail if the slab isn't full anymore due to a cocurrent free.

nit: cocurrent -> concurrent

>  /*
>   * Slow path handling. This may still be called frequently since objects
>   * have a longer lifetime than the cpu slabs in most processing loads.
> @@ -7113,34 +7143,41 @@ __refill_objects_node(struct kmem_cache *s, void **p, gfp_t gfp, unsigned int mi
>  
>  	list_for_each_entry_safe(slab, slab2, &pc.slabs, slab_list) {
>  
> +		unsigned int count;
> +
>  		list_del(&slab->slab_list);
>  
> -		object = get_freelist_nofreeze(s, slab);
> +		object = get_freelist_nofreeze(s, slab, &count);
>  
> -		while (object && refilled < max) {
> +		while (count && refilled < max) {
>  			p[refilled] = object;
>  			object = get_freepointer(s, object);
>  			maybe_wipe_obj_freeptr(s, p[refilled]);
>  
>  			refilled++;
> +			count--;
>  		}
>  
>  		/*
>  		 * Freelist had more objects than we can accommodate, we need to
> -		 * free them back. We can treat it like a detached freelist, just
> -		 * need to find the tail object.
> +		 * free them back. First we try to be optimistic and assume the
> +		 * slab is stil full since we just detached its freelist.

nit: stil -> still

> +		 * Otherwise we must need to find the tail object.

nit: must need to -> must

-- 
Cheers,
Harry / Hyeonggon