Re: [PATCH]: PCI: GART iommu alignment fixes [v2]

[Prarit Bhargava]

Joerg Roedel wrote:
> On Wed, Jul 23, 2008 at 07:47:03PM -0400, Prarit Bhargava wrote:
>   
> > > > Interesting. Have you experienced any problems because of that
> > > > misbehavior in the GART code? AMD IOMMU currently also violates this
> > > > requirement. I will send a patch to fix that there too.
> > > >    
> > > >         
> > >  
> > >       
> > Joerg,  yes I can see misbehavior caused by this code.  O/w I wouldn't 
> > be spending my time fixing it :) :)
> >
> > See below ....
> >
> >     
> > > IIRC, only PARISC and POWER IOMMUs follow the above rule. So I also
> > > wondered what problem he hit.
> > >  
> > >       
> > I wonder if IBM's Calgary IOMMU needs this fix? ... I've added Ed 
> > Pollard to find out.
> >
> > On big memory footprint (16G or above) systems it is possible that the 
> > e820 map reserves most of the lower 4G of memory for system use*.  So 
> > it's possible that the 4G region is almost completely reserved at boot 
> > time and so the kernel starts using the IOMMU for DMA (see 
> > dma_alloc_coherent()).  The addresses returned are not properly aligned, 
> > and this causes serious problems for some drivers that require a 
> > physical aligned address for the device.
> >     
>
> Do you have a list of driver which require this? 

No, I don't have a list. :(

But it seems that the skge driver suffers from this because this code 
exists in the driver:

       skge->mem = pci_alloc_consistent(hw->pdev, skge->mem_size, 
&skge->dma);
       if (!skge->mem)
               return -ENOMEM;

       BUG_ON(skge->dma & 7);

       if ((u64)skge->dma >> 32 != ((u64) skge->dma + skge->mem_size) 
>> 32) {
               printk(KERN_ERR PFX "pci_alloc_consistent region crosses 
4G boundary\n");
               err = -EINVAL;
               goto free_pci_mem;
       }


If pci_alloc_consistent did the "right" thing, we should *never* see 
that warning message.

In theory, any 32-bit device attempting to request larger than PAGE_SIZE 
DMA memory on a system where no memory is available below 4G should show 
this problem.

> I would like to
> reproduce this issue. Does it also happen when you start the kernel with
> iommu=force (GART should then be used for all DMA remapping) too?
>   

Yes, this happens if you specify iommu=force on the command line.

P.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/