Re: mlx5e:zero-prefix corruption during TCP DMA-BUF RX - help requested

[jiabin deng]

Hi Dragos,

I applied your suggested change to `rq_end_pad_mode()` on the receiver
and reran the standalone GPU-memory reproducer I sent previously. With
MTU set to 1500, the single-direction test covered eight message
sizes, with 50 transfers per size, for a total of 400 transfers.

All 400 transfers completed successfully. Full payload comparisons
found zero failing cases, zero incorrect bytes, and zero corrupted
intervals. The zero-prefix corruption observed in the earlier tests
did not recur in this patched run.

Thank you for continuing to investigate this issue and for suggesting
the fix. I also greatly appreciate the time and effort you and the
other maintainers dedicate to the open-source community. As a student,
I am excited to have the opportunity to contribute to the community
and to learn from this process.

I would be happy to help prepare a formal patch with the test results.
Would you prefer to submit the fix yourself, or would it be helpful
for me to prepare and submit it based on your suggested change?

Please also keep me copied on any further discussion or developments
related to this issue.

Best regards,

Jiabin

On Tue, Oct 20, 2026 at 6:56 PM Dragos Tatulea <dtatulea@xxxxxxxxxx> wrote:
>
> Hi Jiabin,
>
> On Thu, Oct 08, 2026 at 02:48:53PM +0800, jiabin deng wrote:
> > Hi Dragos,
> >
> > Thank you for following up and suggesting the CPU-buffer comparison. We have
> > now completed both ordinary CPU TCP and CPU-backed DMA-BUF receive controls,
> > with no payload corruption in either 400-iteration run.
> >
> > Please find attached a standalone GPU-memory reproducer with a README covering
> > the required environment, build instructions, two-host execution, and result
> > collection. It includes a Python TCP sender, a C++ receiver using the CUDA
> > Driver API and Linux device-memory TCP interfaces, and scripts for SHAMPO
> > header tracing and offline byte comparisons.
> >
> Thank you for the reproducer. Unfortunately I didn't get the right HW to
> reproduce this.
>
> Pasting from your previous email:
> > One possible lead is an interaction between SHAMPO end eadding and
> > Relaxed Ordering writes. In this version, rq_end_pad_mode() disables
> > end padding only when LRO is enabled and write RO is supported.
> > SHAMPO therefore selects ALIGN; at the same time, write RO is enabled
> > on the payload MKey when the device advertises write RO support.
> >
> > Commit e2351e517068 ("net/mlx5e: RX, Avoid possible data corruption
> > when relaxed ordering and LRO combined") describes a related issue
> > with consecutively placed LRO payloads. We would like to understand
> > whether SHAMPO might also need to account for a similar situation.
> > We do not yet have results from padding/RO diagnostic experiments.
> Initially I dismissed this commit because it seemed unrelated. But now
> I see that you analysis was probably spot on.
>
> So disabling alignment padding mode is definetly a very useful experiment.
> It would go in the lines of:
>
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
> index 70a855c6d19c..794d8be88f0c 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
> @@ -911,10 +911,11 @@ static void mlx5e_build_rx_cq_param(struct mlx5_core_dev *mdev,
>
>  static u8 rq_end_pad_mode(struct mlx5_core_dev *mdev, struct mlx5e_params *params)
>  {
> -       bool lro_en = params->packet_merge.type == MLX5E_PACKET_MERGE_LRO;
> +       bool no_pad = params->packet_merge.type == MLX5E_PACKET_MERGE_LRO ||
> +                     params->packet_merge.type == MLX5E_PACKET_MERGE_SHAMPO;
>         bool ro = MLX5_CAP_GEN(mdev, relaxed_ordering_write);
>
> -       return ro && lro_en ?
> +       return ro && no_pad ?
>                 MLX5_WQ_END_PAD_MODE_NONE : MLX5_WQ_END_PAD_MODE_ALIGN;
>  }
>
> Could you try this please and let us know if you still encounter the
> issue? In the meantime I am checking about this with other people.
>
> Thanks,
> Dragos