Re: [PATCH] net: make SK_MEMORY_PCPU_RESERV tunable

[Eric Dumazet]

On Mon, Feb 26, 2024 at 3:25 AM Adam Li <adamli@xxxxxxxxxxxxxxxxxxxxxx> wrote:
>
> This patch adds /proc/sys/net/core/mem_pcpu_rsv sysctl file,
> to make SK_MEMORY_PCPU_RESERV tunable.
>
> Commit 3cd3399dd7a8 ("net: implement per-cpu reserves for
> memory_allocated") introduced per-cpu forward alloc cache:
>
> "Implement a per-cpu cache of +1/-1 MB, to reduce number
> of changes to sk->sk_prot->memory_allocated, which
> would otherwise be cause of false sharing."
>
> sk_prot->memory_allocated points to global atomic variable:
> atomic_long_t tcp_memory_allocated ____cacheline_aligned_in_smp;
>
> If increasing the per-cpu cache size from 1MB to e.g. 16MB,
> changes to sk->sk_prot->memory_allocated can be further reduced.
> Performance may be improved on system with many cores.

This looks good, do you have any performance numbers to share ?

On a host with 384 threads, 384*16 ->  6 GB of memory.

With this kind of use, we might need a shrinker...