[PATCH rdma-next 1/2] RDMA/mlx5: Fix undefined shift of user RQ WQE size

From: Edward Srouji

Date: Thu Jun 11 2026 - 08:52:23 EST


From: Maher Sanalla <msanalla@xxxxxxxxxx>

set_rq_size() computes the RQ WQE size as "1 << rq_wqe_shift" based on
the user-provided rq_wqe_shift, which is only checked to be greater than
32, so shifts of 32 are still accepted. A shift of 31 also overflows a
signed integer, leading to undefined behavior.

Use check_shl_overflow() to compute the RQ WQE size and reject any
invalid values.

Fixes: e126ba97dba9 ("mlx5: Add driver for Mellanox Connect-IB adapters")
Signed-off-by: Maher Sanalla <msanalla@xxxxxxxxxx>
Signed-off-by: Edward Srouji <edwards@xxxxxxxxxx>
---
drivers/infiniband/hw/mlx5/qp.c | 11 ++++-------
1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index e8d34d54b43527e0595ec9e2fb93dc7e9bedba92..7674290d0afaf466a6b98cbed86d247ee550bd8d 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -454,16 +454,13 @@ static int set_rq_size(struct mlx5_ib_dev *dev, struct ib_qp_cap *cap,

if (ucmd) {
qp->rq.wqe_cnt = ucmd->rq_wqe_count;
- if (ucmd->rq_wqe_shift > BITS_PER_BYTE * sizeof(ucmd->rq_wqe_shift))
- return -EINVAL;
qp->rq.wqe_shift = ucmd->rq_wqe_shift;
- if ((1 << qp->rq.wqe_shift) /
- sizeof(struct mlx5_wqe_data_seg) <
- wq_sig)
+ if (check_shl_overflow(1, qp->rq.wqe_shift, &wqe_size))
+ return -EINVAL;
+ if (wqe_size / sizeof(struct mlx5_wqe_data_seg) < wq_sig)
return -EINVAL;
qp->rq.max_gs =
- (1 << qp->rq.wqe_shift) /
- sizeof(struct mlx5_wqe_data_seg) -
+ wqe_size / sizeof(struct mlx5_wqe_data_seg) -
wq_sig;
qp->rq.max_post = qp->rq.wqe_cnt;
} else {

--
2.49.0