Re: [PATCH net] net/mlx5e: SHAMPO, Make reserved size independent of page size

From: Tariq Toukan
Date: Thu Mar 27 2025 - 13:59:01 EST




On 25/03/2025 16:04, Simon Horman wrote:
On Sun, Mar 23, 2025 at 02:28:26PM +0200, Tariq Toukan wrote:
From: Lama Kayal <lkayal@xxxxxxxxxx>

When hw-gro is enabled, the maximum number of header entries that are
needed per wqe (hd_per_wqe) is calculated based on the size of the
reservations among other parameters.

Miscalculation of the size of reservations leads to incorrect
calculation of hd_per_wqe as 0, particularly in the case of large page
size like in aarch64, this prevents the SHAMPO header from being
correctly initialized in the device, ultimately causing the following
cqe err that indicates a violation of PD.

Hi Lama, Tariq, all,

If I understand things correctly, hd_per_wqe is calculated
in mlx5e_shampo_hd_per_wqe() like this:

u32 mlx5e_shampo_hd_per_wqe(struct mlx5_core_dev *mdev,
struct mlx5e_params *params, struct mlx5e_rq_param *rq_param)
{
int resv_size = BIT(mlx5e_shampo_get_log_rsrv_size(mdev, params)) * PAGE_SIZE;
u16 num_strides = BIT(mlx5e_mpwqe_get_log_num_strides(mdev, params, NULL));
int pkt_per_resv = BIT(mlx5e_shampo_get_log_pkt_per_rsrv(mdev, params));
u8 log_stride_sz = mlx5e_mpwqe_get_log_stride_size(mdev, params, NULL);
int wqe_size = BIT(log_stride_sz) * num_strides; u32 hd_per_wqe;

/* Assumption: hd_per_wqe % 8 == 0. */
hd_per_wqe = (wqe_size / resv_size) * pkt_per_resv; mlx5_core_dbg(mdev, "%s hd_per_wqe = %d rsrv_size = %d wqe_size = %d pkt_per_resv = %d\n", __func__, hd_per_wqe, resv_size, wqe_size, pkt_per_resv);
return hd_per_wqe;
}

I can see that if PAGE_SIZE was some multiple of 4k, and thus
larger than wqe_size, then this could lead to hd_per_wqe being zero.

But I note that mlx5e_mpwqe_get_log_stride_size() may return PAGE_SHIFT.
And I wonder if that leads to wqe_size being larger than expected by this
patch in cases where the PAGE_SIZE is greater than 4k.

Likewise in mlx5e_shampo_get_log_cq_size(), which seems to have a large overlap
codewise with mlx5e_shampo_hd_per_wqe().



Hi Simon,

Different settings lead to different combinations of num_strides and stride_size. However, they affect each other in a way that the resulting wqe_size has the expected (~preset) value.

In mlx5e_mpwqe_get_log_num_strides() you can see that if stride_size grows, then num_strides decreases accordingly.

In addition, to reduce mistakes/bugs, we have a few WARNs() along the calculations, in addition to a verifier function mlx5e_verify_rx_mpwqe_strides().

Thanks,
Tariq