Re: [PATCH net-next 00/10] net/mlx5e: Add netdev support for data direct

[Mina Almasry]

On Fri, Oct 9, 2026 at 9:42 AM Mina Almasry <almasrymina@xxxxxxxxxx> wrote:
>
> On Thu, Oct 8, 2026 at 6:28 AM Tariq Toukan <tariqt@xxxxxxxxxx> wrote:
> >
> > Hi,
> >
> > This patch series by Dragos adds data direct support for netdev devices,
> > allowing a mlx5 netdev to issue DMA commands through multiple paths.
> > This feature is critical for improving performance and reaching line
> > rate in certain environments where issuing PCI transactions over one
> > path may be significantly faster than over another. These differences
> > can arise from various PCI generations in the system or the specific
> > system topology.
> >
> > Data direct for netdev will work with devmem by returning the data
> > direct DMA device instead of the DMA device of the NIC. Each PF netdev
> > registers its own data direct device and support for RX and TX datapath
> > is added. The feature is selected through a new data direct priv flag.
> >
> > The feature is enabled though a ethtool private flag. When this flag
> > is enabled, the driver will return the data direct device in the
> > ndo_queue_get_dma_dev op.
> >
> > During data_direct device unbind, a re-creation of the channels is
> > triggered.
> >
> > A note about the unbind: it does not does not tear down dmabuf bindings
> > attached to the data direct device; the recreated channels will still
> > pick up the binding via rxq->mp_params and post DMA addresses from the
> > data direct IOMMU domain to the PF, causing IOMMU faults. A devmem
> > revoke hook is needed to close those bindings before the switch. This is
> > out of scope for this series.
> >
>
> Elaborate on this please. Are you referring to the fact that on
> page_pool_scrub we unmap the netmems dma-mapped by the page-pool, but
> we don't unmap the netmems dma-mapped by the memory provider?
>
> What's the impact of leaving this unfixed? Is the machine going to
> crash if the device goes away but we don't unmap the dmabuf?

Oh, you're referring to an even different bug than the one I had in
mind. (I'll see if I can submit a fix for the bug I had in mind).

But in general I think not addressing the issue you're referring to is
way too messy. I think we do indeed need to unbind the dma-buf if the
dma-dev is going away. The invariant should be that the
binding->attachment->dev should not change at all during the entire
lifetime of the binding. We already 'revoke' the dma-buf binding if
the netdev is going away, sorta (there is a bug there I need to fix.
The uninstall function in the provider doesn't actually unmap the
dma-buf :sad face:).

I've worked with my LLM to suggest some changes that (we think) make
this work, but you may disagree. I'll post the suggestions. But please
I think this should be handled one way or another.

-- 
Thanks,
Mina