[PATCH v1] dma-contiguous: try local node first for dma_alloc_contiguous()

[Feng Tang]

There was a bug report on a multi-numa-nodes ARM server that when
IOMMU is disabled, the dma_alloc_coherent() function always returns
memory from node 0 even for devices attaching to other nodes, while
they can get local dma memory when IOMMU is on with the same API.

The reason is, when IOMMU is disabled, the dma_alloc_coherent() will
go the direct way and call dma_alloc_contiguous(). The system doesn't
have any explicit cma setting (like per-numa cma), and only has a
default 64MB cma reserved area (on node 0), where kernel will try
first to allocate memory from.

Make the dma allocation more locality friendly by trying first the
numa aware allocation alloc_pages_node() before falling back to the
reserved cma area.

One more thought is to check the node of the reserved cma area and
only call alloc_pages_nodes() when it isn't the same node that the
device attaches to.

Signed-off-by: Feng Tang <feng.tang@xxxxxxxxxxxxxxxxx>
---
 kernel/dma/contiguous.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
index c56004d314dc..0180f40f094e 100644
--- a/kernel/dma/contiguous.c
+++ b/kernel/dma/contiguous.c
@@ -371,8 +371,9 @@ static struct page *cma_alloc_aligned(struct cma *cma, size_t size, gfp_t gfp)
  */
 struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
 {
-#ifdef CONFIG_DMA_NUMA_CMA
+#ifdef CONFIG_NUMA
 	int nid = dev_to_node(dev);
+	struct page *page;
 #endif
 
 	/* CMA can be used only in the context which permits sleeping */
@@ -386,7 +387,6 @@ struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
 #ifdef CONFIG_DMA_NUMA_CMA
 	if (nid != NUMA_NO_NODE && !(gfp & (GFP_DMA | GFP_DMA32))) {
 		struct cma *cma = dma_contiguous_pernuma_area[nid];
-		struct page *page;
 
 		if (cma) {
 			page = cma_alloc_aligned(cma, size, gfp);
@@ -402,6 +402,14 @@ struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
 		}
 	}
 #endif
+
+#ifdef CONFIG_NUMA
+	/* Try first to allocate memory on the same node as the device */
+	page = alloc_pages_node(nid, gfp, get_order(size));
+	if (page)
+		return page;
+#endif
+
 	if (!dma_contiguous_default_area)
 		return NULL;
 
-- 
2.39.5 (Apple Git-154)