Re: [PATCH v9 net] octeontx2-af: switch qmem from coherent DMA alloc to streaming DMA mapping

[Leon Romanovsky]

On Tue, Sep 15, 2026 at 02:17:50PM +0530, Ratheesh Kannoth wrote:
> On 2026-09-15 at 13:46:33, Leon Romanovsky (leon@xxxxxxxxxx) wrote:
> > On Fri, Sep 11, 2026 at 07:56:49AM +0530, Ratheesh Kannoth wrote:
> > > qmem_alloc() uses dma_alloc_attrs() with DMA_ATTR_FORCE_CONTIGUOUS, which
> > > allocates CPU-cache-coherent DMA memory and, with CMA enabled, draws from
> > > the CMA pool. qmem backs NIX/NPA queue contexts, admin queues, and LMTST
> > > regions (including CN10K LMTST areas that span page boundaries), so
> > > consumption grows with enabled interfaces and is hard to provision in CMA.
> > >
> > > Switch qmem to a streaming-DMA-style path: allocate memory with kmalloc(),
> > > then map it for device access with dma_map_single(). Add
> > > otx2_dma_alloc_coherent() and otx2_dma_free_coherent() helpers and wire
> > > qmem_alloc()/qmem_free() through them instead of dma_alloc_attrs()/
> > > dma_free_attrs().
> > >
> > > This works on Octeon because the octeontx2 driver is written for
> > > DMA-coherent devices: Octeon platforms provide IO coherency (via SMMU), so
> > > the driver already uses streaming DMA APIs for packet data while
> > > deliberately skipping explicit CPU cache sync (DMA_ATTR_SKIP_CPU_SYNC).
> > > The same IO coherency lets qmem use a streaming map of kmalloc-backed
> > > memory instead of a dedicated coherent allocator or CMA reservation. That
> > > is valid because the platform is DMA-coherent, not because omitting
> > > dma_sync_* magically makes memory coherent.
> > >
> > > cc: Leon Romanovsky <leon@xxxxxxxxxx>
> > > Fixes: 73d33dbc0723 ("octeontx2-af: Use DMA_ATTR_FORCE_CONTIGUOUS attribute in DMA alloc")
> >
> > The code itself looks fine now, but the commit message does not explain
> > why this patch is needed. It only describes what the patch does, which is
> > unnecessary here since the change itself is straightforward.
> >
> > Also, no bug is described, so neither the Fixes tag nor the "net" target
> > is appropriate for this patch.
> >
> 
> Below commit message is Okay ?

How about a simpler message?

octeontx2-af: Fix memory scaling limitation in SR-IOV mode

The original code used DMA_ATTR_FORCE_CONTIGUOUS, which could exhaust
the CMA pool when a large number of VFs were requested.

Fix this by switching to the DMA streaming API. This is equivalent on
Octeon platforms, which provide full I/O coherency via the SMMU.

Thanks

> 
> octeontx2-af: switch qmem from coherent DMA alloc to streaming DMA mapping
> 
> On platforms with CMA enabled, qmem_alloc() silently drains the CMA pool.
> qmem_alloc() calls dma_alloc_attrs() with DMA_ATTR_FORCE_CONTIGUOUS, which
> routes every allocation through the CMA allocator. qmem backs NIX/NPA queue
> contexts, admin queues, and LMTST regions (including CN10K LMTST areas that
> span page boundaries), so the total footprint scales with the number of
> enabled interfaces and SR-IOV VFs. On systems with many active interfaces or
> many VF counts, this exhausts the CMA pool and causes qmem_alloc() to fail
> at interface bring-up or probe time
> 
> Octeon platforms provide full IO coherency via SMMU, so the driver already
> uses streaming DMA APIs for packet data while deliberately skipping explicit
> CPU cache sync (DMA_ATTR_SKIP_CPU_SYNC). The same IO coherency means qmem
> does not need a dedicated coherent allocator or CMA reservation — a
> streaming map of ordinary kmalloc-backed memory works correctly.
> 
> Fix this by switching qmem to a streaming-DMA path: allocate memory with
> kzalloc(), then map it for device access with dma_map_single(). Add
> otx2_dma_alloc_coherent() and otx2_dma_free_coherent() helpers and wire
> qmem_alloc()/qmem_free() through them instead of dma_alloc_attrs()/
> dma_free_attrs(). This is valid because the platform is DMA-coherent, not
> because omitting dma_sync_* magically makes memory coherent.
> 
> Fixes: 73d33dbc0723 ("octeontx2-af: Use DMA_ATTR_FORCE_CONTIGUOUS attribute in DMA alloc")
> Signed-off-by: Ratheesh Kannoth rkannoth@xxxxxxxxxxx