Re: [PATCH] net/mlx5: allocate dma pool structs on the pool's NUMA node

[Tariq Toukan]

On 05/10/2026 16:46, Seongjun Hong wrote:
> The mlx5 DMA pool structs (mlx5_dma_pool, mlx5_dma_pool_page and its
> block bitmap, and mlx5_frag_buf_node_pools) are allocated without a
> node hint, so they land on the node of whichever CPU happens to create
> or fill the pool, whiile 

nit: typo

> the DMA pages they describe are allocated on
> the pool's node.
>
> The pool, page and bitmap are dereferenced on every block allocation
> and free. Allocate them on the pool's NUMA node as well, so that all
> of a pool's state lives on one node.

Thanks for the patch.

That is true, but these structs are only used from the control path.
That is once per WQ/CQ/EQ or doorbell record at open/close, not per packet.

I doubt we can measure any benefit, so it's just a matter of consistency 
with the other fields access in data-path.

The API kzalloc_node falls back if node is offline, so it's safe.

No strong motivation, but we can accept it.
Please fix the typo and emphasize that these are control-path objects.

> Signed-off-by: Seongjun Hong <hsj0512@xxxxxxxxx>
> ---
>   drivers/net/ethernet/mellanox/mlx5/core/alloc.c | 11 ++++++-----
>   1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/alloc.c b/drivers/net/ethernet/mellanox/mlx5/core/alloc.c
> index a92cf545bdaf..dcd281c4691d 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/alloc.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/alloc.c
> @@ -110,7 +110,7 @@ static struct mlx5_dma_pool *mlx5_dma_pool_create(struct mlx5_core_dev *dev,
>   {
>   	struct mlx5_dma_pool *pool;
>   
> -	pool = kzalloc_obj(*pool);
> +	pool = kzalloc_node(sizeof(*pool), GFP_KERNEL, node);
>   	if (!pool)
>   		return NULL;
>   
> @@ -127,19 +127,20 @@ mlx5_dma_pool_page_alloc(struct mlx5_dma_pool *pool)
>   {
>   	int blocks_per_page = BIT(PAGE_SHIFT - pool->block_shift);
>   	struct mlx5_dma_pool_page *page;
> +	int node = pool->node;
>   
> -	page = kzalloc_obj(*page);
> +	page = kzalloc_node(sizeof(*page), GFP_KERNEL, node);
>   	if (!page)
>   		goto err_out;
>   
>   	page->pool = pool;
> -	page->bitmap = bitmap_zalloc(blocks_per_page, GFP_KERNEL);
> +	page->bitmap = bitmap_zalloc_node(blocks_per_page, GFP_KERNEL, node);
>   	if (!page->bitmap)
>   		goto err_free_page;
>   
>   	bitmap_fill(page->bitmap, blocks_per_page);
>   	page->buf = mlx5_dma_zalloc_coherent_node(pool->dev, PAGE_SIZE,
> -						  &page->dma, pool->node);
> +						  &page->dma, node);
>   	if (!page->buf)
>   		goto err_free_bitmap;
>   
> @@ -278,7 +279,7 @@ mlx5_frag_buf_node_pools_create(struct mlx5_core_dev *dev, int node)
>   {
>   	struct mlx5_frag_buf_node_pools *node_pools;
>   
> -	node_pools = kzalloc_obj(*node_pools);
> +	node_pools = kzalloc_node(sizeof(*node_pools), GFP_KERNEL, node);
>   	if (!node_pools)
>   		return NULL;
>   
>
> ---
> base-commit: 8b4e7209c842d8cb9516f1f5ef0a88aa2d8831a6
> change-id: 20261005-net-mlx5-numa-allocate-pool-page-2ee20a04147e
>
> Best regards,