Re: [PATCH V2] mm/slab: introduce kvfree_rcu_barrier_on_cache() for cache destruction

[Harry Yoo]

On Tue, Dec 02, 2025 at 07:16:26PM +0900, Harry Yoo wrote:
> Currently, kvfree_rcu_barrier() flushes RCU sheaves across all slab
> caches when a cache is destroyed. This is unnecessary; only the RCU
> sheaves belonging to the cache being destroyed need to be flushed.
> 
> As suggested by Vlastimil Babka, introduce a weaker form of
> kvfree_rcu_barrier() that operates on a specific slab cache.
> 
> Factor out flush_rcu_sheaves_on_cache() from flush_all_rcu_sheaves() and
> call it from flush_all_rcu_sheaves() and kvfree_rcu_barrier_on_cache().
> 
> Call kvfree_rcu_barrier_on_cache() instead of kvfree_rcu_barrier() on
> cache destruction.
> 
> The performance benefit is evaluated on a 12 core 24 threads AMD Ryzen
> 5900X machine (1 socket), by loading slub_kunit module.
> 
> Before:
>   Total calls: 19
>   Average latency (us): 18127
>   Total time (us): 344414
> 
> After:
>   Total calls: 19
>   Average latency (us): 10066
>   Total time (us): 191264
> 
> Two performance regression have been reported:
>   - stress module loader test's runtime increases by 50-60% (Daniel)

So I took a look at why this regression is fixed. I didn't expect this
is going to be fixed because Daniel said CONFIG_CODE_TAGGING is enabled,
and there is still a heavy kvfree_rcu_barrier() call during module unloading.

As Vlastimil pointed out off-list, there should be kmem_cache_destroy()
calls somewhere.

So I ran kmod.sh and traced kmem_cache_destroy() calls:
> === kmem_cache_destroy Latency Statistics ===
> Total calls: 6346
> Average latency (us): 5156
> Total time (us): 32725981

Oh, it's called 6346 times during the test? That's impressive.
It also spent 32.725 seconds just for kmem_cache_destroy(), out of total
runtime of 96 seconds.

> === Top 2 stack traces involving kmem_cache_destroy ===
> 
> @stacks[
>     kmem_cache_destroy+1
>     cleanup_module+118
>     __do_sys_delete_module.isra.0+451
>     __x64_sys_delete_module+18
>     x64_sys_call+7366
>     do_syscall_64+128
>     entry_SYSCALL_64_after_hwframe+118
> ]: 1840

It seems tools/testing/selftests/kmod/kmod.sh is using xfs module for testing
and it creates & destroys many slab caches. (see exit_xfs_fs() ->
xfs_destroy_caches()).

Mystery solved, I guess :D

> @stacks[
>     kmem_cache_destroy+1
>     rcbagbt_init_cur_cache+4219734
>     __do_sys_delete_module.isra.0+451
>     __x64_sys_delete_module+18
>     x64_sys_call+7366
>     do_syscall_64+128
>     entry_SYSCALL_64_after_hwframe+118
> ]: 1955

I don't get this one though. Why is the rcbagbt init function (also
from xfs) called during module unloading?

>   - internal graphics test's runtime on Tegra23 increases by 35% (Jon)
> 
> They are fixed by this change.
> 
> Suggested-by: Vlastimil Babka <vbabka@xxxxxxx>
> Fixes: ec66e0d59952 ("slab: add sheaf support for batching kfree_rcu() operations")
> Cc: <stable@xxxxxxxxxxxxxxx>
> Link: https://lore.kernel.org/linux-mm/1bda09da-93be-4737-aef0-d47f8c5c9301@xxxxxxx
> Reported-and-tested-by: Daniel Gomez <da.gomez@xxxxxxxxxxx>
> Closes: https://lore.kernel.org/linux-mm/0406562e-2066-4cf8-9902-b2b0616dd742@xxxxxxxxxx
> Reported-and-tested-by: Jon Hunter <jonathanh@xxxxxxxxxx>
> Closes: https://lore.kernel.org/linux-mm/e988eff6-1287-425e-a06c-805af5bbf262@xxxxxxxxxx
> Signed-off-by: Harry Yoo <harry.yoo@xxxxxxxxxx>
> ---

-- 
Cheers,
Harry / Hyeonggon