Re: iommu dma mapping alignment requirements

From: Steve Wise
Date: Thu Dec 20 2007 - 17:12:57 EST


Benjamin Herrenschmidt wrote:
On Thu, 2007-12-20 at 15:02 -0600, Steve Wise wrote:
Benjamin Herrenschmidt wrote:
Adding A few more people to the discussion. You may well be right and we
would have to provide the same alignment, though that sucks a bit as one
of the reason we switched to 4K for the IOMMU is that the iommu space
available on pSeries is very small and we were running out of it with
64K pages and lots of networking activity.

But smarter NIC drivers can resolve this too, I think, but perhaps carving up full pages of mapped buffers instead of just assuming mapping is free...

True, but the problem still happenens today, if we switch back to 64K
iommu page size (which should be possible, I need to fix that), we
-will- run out of iommu space on typical workloads and that is not
acceptable.

So we need to find a compromise.

What I might do is something around the lines of: If size >= PAGE_SIZE,
and vaddr (page_address + offset) is PAGE_SIZE aligned, then I enforce
alignment of the resulting mapping.

That should fix your case. Anything requesting smaller than PAGE_SIZE
mappings would lose that alignment but I -think- it should be safe, and
you still always get 4K alignment anyway (+/- your offset) so at least
small alignment restrictions are still enforced (such as cache line
alignment etc...).

I'll send you a test patch later today.

Ben.


Sounds good. Thanks!

Note, that these smaller sub-host-page-sized mappings might pollute the address space causing full aligned host-page-size maps to become scarce... Maybe there's a clever way to keep those in their own segment of the address space?


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/