[PATCH rdma-next v2 0/3] RDMA: Fix DEVX UMEM mailbox size overflow
From: Edward Srouji
Date: Wed Nov 11 2026 - 09:05:23 EST
Registering a DEVX UMEM builds a CREATE_UMEM mailbox whose size grows
with the number of MTTs, i.e. with the user-supplied length and
pgsz_bitmap. The size is stored in a u32, so a UMEM of about 2TB with
4KB pages wraps it, the mailbox is under-allocated and
mlx5_ib_populate_pas() writes past its end.
Patch 1 fixes a pre-existing issue in the same path, reported by Sashiko
on v1: devx_umem_find_best_pgsize() can return a page size that is not
in the user's pgsz_bitmap.
Patch 2 computes the mailbox size with the saturating size_mul() and
size_add() helpers and rejects UMEMs whose mailbox exceeds INT_MAX, the
largest in_size mlx5_cmd_exec() accepts.
Patch 3 handles the remaining corner: _uverbs_alloc() adds a small
header to the request before calling kvmalloc(), so a mailbox just
under INT_MAX can still push kvmalloc() over its INT_MAX limit and
trigger a WARN. Since these sizes come from userspace, the allocation
now uses __GFP_NOWARN and fails with -ENOMEM.
Mailbox sizes within 512 bytes of INT_MAX also overflow the block count
in mlx5_calc_cmd_blocks(). That predates this series and fails safely
with -ENOMEM before the command reaches firmware, so it is left for a
separate mlx5_core change.
The overflow was found with an AI-assisted static analysis tool and
confirmed by code review.
Patch 2 relies on size_mul() and size_add(), so stable trees older than
5.18 will need a manual backport.
Signed-off-by: Edward Srouji <edwards@xxxxxxxxxx>
---
Changes in v2:
- Keep cmd->inlen unsigned and bound the mailbox size explicitly
- Add patch 1 to respect pgsz_bitmap when reducing the page size
(Sashiko)
- Link to v1: https://lore.kernel.org/r/20261109-devx-umem-mailbox-overflow-v1-0-4a63baab25dc@xxxxxxxxxx
---
Edward Srouji (1):
RDMA/mlx5: Respect pgsz_bitmap when reducing the DEVX UMEM page size
Uri Dar (2):
RDMA/mlx5: Fix DEVX UMEM mailbox size truncation
RDMA/core: Don't warn on oversized bundle allocations
drivers/infiniband/core/ib_core_uverbs.c | 2 +-
drivers/infiniband/hw/mlx5/devx.c | 23 ++++++++++++++++-------
2 files changed, 17 insertions(+), 8 deletions(-)
---
base-commit: e50525a486300e5ff8014c9985b8477b2fca0368
change-id: 20261109-devx-umem-mailbox-overflow-c741bc73deb4
Best regards,
--
Edward Srouji <edwards@xxxxxxxxxx>