Re: [PATCH net-next 0/4] bpf: bpf_htab: Add BPF_MAP_TYPE_PERCPU_HASH

[Alexei Starovoitov]

On Tue, Jan 12, 2016 at 01:48:10PM +0800, Ming Lei wrote:
> The total memory consumption is still much less than memory consumed
> by percpu hash since a new element is only added to hash if the key is run
> on that CPU. Most of times, for one key it may touch very few CPUs.
> 
> For percpu hash, the memory is always allocated to every CPU no matter
> if the key is run from the CPU.

In Martin's use case all cpus are servicing network traffic and all of them
are counting packets.

> In my test, removing the current kmalloc() in update element callback can
> improve io thoughput by 10% not mention the percpu ida allocation cost, and
> looks it isn't cheap. That is why I don't think it is good to
> introduce another new
> allocation in the eBPF prog path.

I don't think anyone is arguing that pre-allocation is not needed.
In some cases better performance can be achieved with pre-allocation,
in some other cases regular hash map will be enough,
and in others hash map with per-cpu is needed as well.

> You can find my test in the link below:
> 
>        https://patchwork.ozlabs.org/patch/556926/

yes, for tools/biolatency pre-allocation is a win,
but in many other cases we simply cannot pre-allocate all elements.