Re: Question about iommu on x86_64 and radeon driver.

From: Daniel Haid
Date: Tue May 31 2011 - 11:34:39 EST


Noo.. It does, but the normal assumption of 'phys_to_virt' ==
'phys_to_bus' is
not valid anymore. Think of a buffer (swiotlb) which has a pool
of pages and when a PCI device wants a page, it hands one out. It also has
other functionality such as 'mapping' of an already allocated page. If the
PCI device asks the IOMMU (swiotlb) to map a page (and if you have
'swiotlb=force'
the page provided has been allocated above 4GB and the device can only handle
up to 32-bit),

Does the radeon driver allocate without DMA32, possibly above 4GB, ...

then swiotlb gives out a page from its own pool. You now have
two addresses: the one from the PCI pool (swiotlb) and the one you already
allocated.

... or does it allocate under 4GB but nevertheless get a page from
the swiotlb pool?

You are suppose to program your PCI card to read/write data to the
page provided from the IOMMU (so the swiotlb), which means that it won't
write to the page you had allocated. Hence there are a calls, such as
'sync_page'..
which will copy the contents from the swiotlb page to the one you had
allocated
(or vice-versa). This is called 'bounce buffer'.

The radeon (and nouveau) don't have the code to call the 'sync_page', and
you wouldn't really want to do so - as it slows down the performance of the
machine. There exists another mechanism which is to allocate the pages
at the start, and not do mapping later one.

Why can the radeon not simply allocate addresses under 4GB and not request
adresses from the iommu at all?

I assume that if you request a page from the IOMMU, you are required to do
these sync_page calls (and that they get optimized away with a hardware IOMMU?).

So if the radeon uses the IOMMU but does not call sync_page even if required to
the code seems to be broken. If this is indeed the case would it not be possible to
simply add the sync_page calls for now (and thus fix the code), if it is not
difficult, and implement the method with more performance later?

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/