Re: Scatter-gather segment merges by IOMMU?

[FUJITA Tomonori]

On Fri, 08 Aug 2008 23:21:28 +0200
Stefan Richter <stefanr@xxxxxxxxxxxxxxxxx> wrote:

> Grant Grundler wrote:
> > On Fri, Aug 8, 2008 at 12:44 PM, Stefan Richter
> > <stefanr@xxxxxxxxxxxxxxxxx> wrote:
> >> Hi all,
> >>
> >> the block layer usually tries to merge s/g segments if consecutive segments
> >> combined fit into the queue's max_segment_size.  When such a scatter gather
> >> list is DMA-mapped, can it happen that an IOMMU collapses the elements even
> >> further, so that sg_dma_len() of a DMA-mapped s/g segment exceeds
> >> max_segment_size?
> > 
> > I don't see how. The IOMMU code only collapses the "physical" mappings and
> > does not add new elements to the SG list. ergo sg_dma_len() shouldn't change.
> > 
> > grant
> > 
> >> As I understood some discussions in the past, this could indeed happen,
> >> which is a nuisance.  But I may have misunderstood something, or something
> >> may have changed in the meantime...
> 
> Well, I'm just doing my homework and am tracking down the various 
> dma_map_sg implementations.
> 
> Here is what PPC does:
> http://lxr.linux.no/linux+v2.6.26/arch/powerpc/kernel/iommu.c#L270
> 
> It looks at dma_get_max_seg_size(dev); and then merges according to it.

Yes, IOMMUs were not able to handle this issue but they should now.


> That's all nice and well, but in my case (FireWire storage protocol 
> a.k.a. SBP-2, which is basically remote DMA), the max_segment_size of 
> the PCI device is different from the size limit of the protocol.  We 
> currently have to deconstruct such merges in the sbp2 drivers again:
> http://lxr.linux.no/linux+v2.6.26/drivers/firewire/fw-sbp2.c#L1384
> 
> Either I keep it that way, or I let the protocol driver manipulate the 
> FireWire controller's dev->dma_parms->max_segment_size (via 
> dma_set_max_seg_size() of course), which is not entirely correct.

Why is it not correct? Device drivers can set
dma_set_max_seg_size(). SCSI does that (see __scsi_alloc_queue).


> I first wanted to use blk_queue_max_segment_size(), but that falls short 
> with dma_map_sg implementations like the above are at work.

Yeah, we store the same value at two places (request_queue and device)
though it's not nice.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/