Re: [RFC PATCH v1] devres: align devres.data strictly only for devm_kmalloc()

[Robin Murphy]

On 2019-12-20 5:13 pm, Peter Zijlstra wrote:
> On Fri, Dec 20, 2019 at 03:01:03PM +0000, Robin Murphy wrote:
> > On 2019-12-20 2:06 pm, Peter Zijlstra wrote:
>
> > > 	data = kmalloc(size + sizeof(struct devres), GFP_KERNEL);

[ afterthought: size could also legitimately be smaller than a pointer 
or some odd length such that the necessary alignment of struct devres 
itself isn't met ]

> > At this point, you'd still need to special-case devm_kmalloc() to ensure
> > size is rounded up to the next ARCH_KMALLOC_MINALIGN granule, or you'd go
> > back to the original problem of the struct devres fields potentially sharing
> > a cache line with the data buffer. That needs to be avoided, because if the
> > devres list is modified while the buffer is mapped for noncoherent DMA
> > (which could legitimately happen as they are nominally distinct allocations
> > with different owners) there's liable to be data corruption one way or the
> > other.
>
> Wait up, why are you allowing non-coherent DMA at less than page size
> granularity? Is that really sane? Is this really supported behaviour for
> devm ?

There are two DMA APIs - the coherent (or "consistent") API for 
allocating buffers which are guaranteed safe for random access at any 
time *is* page-based, and on non-coherent architectures typically 
involves a non-cacheable remap. There is also the streaming API for 
one-off transfers of data already existing at a given kernel address 
(think network packets, USB URBs, etc), which on non-coherent 
architectures is achieved with explicit cache maintenance plus an API 
contract that buffers must not be explicitly accessed by CPUs for the 
duration of the mapping. Addresses from kmalloc() are explicitly valid 
for dma_map_single() (and indeed are about the only thing you'd ever 
reasonably feed it), which is the primary reason why 
ARCH_KMALLOC_MINALIGN gets so big on architectures which can be 
non-coherent and also suffer from creative cache designs.

See DMA-API.txt and DMA-API-HOWTO.txt in Documentation/ for more details 
if you like.

Robin.