That sounds like a good way to make AGP accesses slower. :)
Seriously, given that AGP is a technology that is being superseded by
PCI Express, I think it's reasonable to look at the range of current
implementations to see what we have to cope with. So I don't think
it's worth worrying too much about the possibility of GARTs that go
through the IOMMU. However, the idea of having phys_to_agp/agp_to_phys
(or virt_to_agp/agp_to_virt) sounds like it wouldn't be too much
effort, if it would help Xen.