[PATCH v7 0/2] RDMA/rxe: fix TOCTOU races in send WQE processing
From: Tristan Madani
Date: Fri Oct 30 2026 - 11:20:22 EST
The SoftRoCE driver maps its send work queue into userspace for direct
posting. The kernel reads WQE fields directly from this shared mapping,
which allows a concurrent userspace thread to modify fields between
kernel reads -- classic TOCTOU.
This series copies each send WQE to a kernel-private buffer before
processing, in both the requester (patch 1) and completer (patch 2)
paths.
v7 addresses all findings from the v6 sashiko AI review and Zhu Yanjun
review:
- FORTIFY_SOURCE: memcpy destination is now the enclosing struct
(send_wqe/comp_wqe) rather than the inner wqe member, so
__member_size() sees the full buffer including the trailing SGE
array.
- Copy size: uses max(max_sge * sizeof(ib_sge), max_inline) to
match the actual queue element data area, ensuring all SGE entries
are captured even when max_sge * sizeof(ib_sge) exceeds max_inline.
- IB_WR_REG_MR: opcode bound raised from IB_WR_ATOMIC_WRITE to
IB_WR_REG_MR so kernel ULP fast registration is not rejected.
Invalid opcodes get mask=0 in the requester (normal error path)
or COMPST_ERROR in the completer.
- num_sge: the requester no longer silently clamps num_sge, letting
the existing rxe_requester() validation detect and error the QP.
The completer validates num_sge and cur_sge bounds, returning
COMPST_ERROR on violation to prevent OOB SGE walks.
- DMA progress writeback: both paths now write back dma.resid,
dma.cur_sge, and dma.sge_offset so retries resume from the correct
offset (fixing IB_WR_FLUSH and partial READ retry regressions).
- State TOCTOU in completer: the smp_load_acquire()d state is
restored after memcpy to prevent a concurrent write from overwriting
it.
- Posted WQE: get_wqe() sets *wqe_p = NULL for posted state,
preventing the timeout_retry path from operating on an uncopied
shared WQE.
- Retry: comp_wqe_valid is invalidated when a retry is initiated,
ensuring retried responses get a fresh copy with updated PSNs.
Tristan Madani (2):
RDMA/rxe: copy send WQE to kernel buffer before processing
RDMA/rxe: copy send WQE to kernel buffer in completer path
drivers/infiniband/sw/rxe/rxe_comp.c | 72 ++++++++++++++++++++++++++++++++++-
drivers/infiniband/sw/rxe/rxe_qp.c | 2 ++
drivers/infiniband/sw/rxe/rxe_req.c | 64 +++++++++++++++++++++++++++++---
drivers/infiniband/sw/rxe/rxe_verbs.h | 12 ++++++
4 files changed, 143 insertions(+), 7 deletions(-)
Tristan Madani (2):
RDMA/rxe: copy send WQE to kernel buffer before processing
RDMA/rxe: copy send WQE to kernel buffer in completer path
drivers/infiniband/sw/rxe/rxe_comp.c | 72 ++++++++++++++++++++++++++-
drivers/infiniband/sw/rxe/rxe_qp.c | 2 +
drivers/infiniband/sw/rxe/rxe_req.c | 64 ++++++++++++++++++++++--
drivers/infiniband/sw/rxe/rxe_verbs.h | 12 +++++
4 files changed, 143 insertions(+), 7 deletions(-)
--
2.47.3