Re: [PATCH] Revert "arm64: Increase the max granular size"

[Chalamarla, Tirumalesh]

On 3/21/16, 10:33 AM, "Catalin Marinas" <catalin.marinas@xxxxxxx> wrote:

>On Mon, Mar 21, 2016 at 05:23:01PM +0000, Will Deacon wrote:
>> On Mon, Mar 21, 2016 at 05:14:03PM +0000, Catalin Marinas wrote:
>> > diff --git a/arch/arm64/include/asm/cache.h b/arch/arm64/include/asm/cache.h
>> > index 5082b30bc2c0..4b5d7b27edaf 100644
>> > --- a/arch/arm64/include/asm/cache.h
>> > +++ b/arch/arm64/include/asm/cache.h
>> > @@ -18,17 +18,17 @@
>> >  
>> >  #include <asm/cachetype.h>
>> >  
>> > -#define L1_CACHE_SHIFT		7
>> > +#define L1_CACHE_SHIFT		6
>> >  #define L1_CACHE_BYTES		(1 << L1_CACHE_SHIFT)
>> >  
>> >  /*
>> >   * Memory returned by kmalloc() may be used for DMA, so we must make
>> > - * sure that all such allocations are cache aligned. Otherwise,
>> > - * unrelated code may cause parts of the buffer to be read into the
>> > - * cache before the transfer is done, causing old data to be seen by
>> > - * the CPU.
>> > + * sure that all such allocations are aligned to the maximum *known*
>> > + * cache line size on ARMv8 systems. Otherwise, unrelated code may cause
>> > + * parts of the buffer to be read into the cache before the transfer is
>> > + * done, causing old data to be seen by the CPU.
>> >   */
>> > -#define ARCH_DMA_MINALIGN	L1_CACHE_BYTES
>> > +#define ARCH_DMA_MINALIGN	(128)
>> 
>> Does this actually fix the reported iperf regression? My assumption was
>> that ARCH_DMA_MINALIGN is the problem, but I could be wrong.
>
>I can't tell. But since I haven't seen any better explanation in this
>thread yet, I hope that at least someone would try this patch and come
>back with numbers.
>
>For networking, SKB_DATA_ALIGN() uses SMP_CACHE_BYTES (== L1_CACHE_BYTES).
>I think (hope) this alignment is not meant for non-coherent DMA,
>otherwise using SMP_CACHE_BYTES wouldn't make sense.

As I see the problem, may be it is because of fewer number of buffers available because of this alignment requirement.
But that should be fixed by making slab alignment a run time thing. I may be totally wrong. 

We are still coming up with a decent benchmark that shows degradation.  
>
>-- 
>Catalin