Re: [Question] devm_kmalloc() for DMA ?

From: Masahiro Yamada
Date: Wed Mar 08 2017 - 22:27:02 EST

Hi Russell, Lars-Peter,

Thanks for your expert comments.

2017-03-09 6:33 GMT+09:00 Lars-Peter Clausen <lars@xxxxxxxxxx>:
> On 03/08/2017 10:19 PM, Russell King - ARM Linux wrote:
>> On Wed, Mar 08, 2017 at 09:44:17PM +0100, Lars-Peter Clausen wrote:
>>> On 03/08/2017 08:59 PM, Russell King - ARM Linux wrote:
>>>> On Wed, Mar 08, 2017 at 08:48:31PM +0100, Lars-Peter Clausen wrote:
>>>>> When the DMA memory is mapped for reading from the device the associated
>>>>> cachelines are invalidated without writeback. There is no guarantee that
>>>>> the changes made to the devres_node have made it to main memory yet, or
>>>>> is there?
>>>> That is incorrect.
>>>> Overlapping cache lines are always written back on transitions from CPU
>>>> to device ownership of the buffer (eg, dma_map_*().)
>>> On ARM. But my understanding is that this is not a universal requirement
>>> according to DMA-API.txt. It says that mappings must be cache line aligned
>>> and otherwise behavior is undefined.
>> There is no use of the term "undefined" in the document you refer to.
>> There is the recommendation that regions are cache line aligned, but
>> there is quite a bit of history in the kernel where DMA has been to
>> regions that are not cache line aligned, and where the DMA region
>> overlaps with data that has recent accesses made to it.
> I says: "Warnings: Memory coherency operates at a granularity called the
> cache line width. In order for memory mapped by this API to operate
> correctly, the mapped region must begin exactly on a cache line
> boundary and end exactly on one." That doesn't sound like a recommendation
> to me. "should" usually implies a recommendation while "must" indicates a
> hard requirement.
> I believe e.g. MIPS will align the address by masking the lower bits off,
> without any flushes. Wouldn't be surprised if other architectures do the same.
>> The situation is improving (in that DMA buffers are being allocated
>> separately, rather than being part of some other structure) but that
>> doesn't mean that it's safe to assume that overlapping cache lines can
>> be invalidated.
>> In any case, DMA with devm allocated buffers really is not a good idea.
> I very much agree with that part.

I understood devm_k*alloc() is not good for DMA.

Now I am tackling on improvement of drivers/mtd/nand/denali.c
and I'd like to make the situation better.

I thought 3 choices.

Is there anything recommended, or should be avoided?

(a) Change devm_kmalloc() to return address aligned with ARCH_DMA_MINALIGN.

(b) Give alignment in your driver if you still want to use devm_.

denali->buf = devm_kmalloc(dev, bufsize + ARCH_DMA_MINALIGN,
if (!denali->buf) {

denali->buf = PTR_ALIGN(denali->buf, ARCH_DMA_MINALIGN);

(c) Use kmalloc() and kfree(). (be careful for memory leak)


Best Regards
Masahiro Yamada