Re: __GFP_REPEAT usage in fq_alloc_node

[Vlastimil Babka]

On 01/06/2017 06:08 PM, Eric Dumazet wrote:
> On Fri, Jan 6, 2017 at 8:55 AM, Vlastimil Babka <vbabka@xxxxxxx> wrote:
>> On 01/06/2017 05:48 PM, Eric Dumazet wrote:
>>> On Fri, Jan 6, 2017 at 8:31 AM, Vlastimil Babka <vbabka@xxxxxxx> wrote:
>>>
>>>>
>>>> I wonder what's that cause of the penalty (when accessing the vmapped
>>>> area I suppose?) Is it higher risk of collisions cache misses within the
>>>> area, compared to consecutive physical adresses?
>>>
>>> I believe tests were done with 48 fq qdisc, each having 2^16 slots.
>>> So I had 48 blocs,of 524288 bytes.
>>>
>>> Trying a bit harder at setup time to get 128 consecutive pages got
>>> less TLB pressure.
>>
>> Hmm that's rather surprising to me. TLB caches the page table lookups
>> and the PFN's of the physical pages it translates to shouldn't matter -
>> the page tables will look the same. With 128 consecutive pages could
>> manifest the reduced collision cache miss effect though.
>>
> 
> To be clear, the difference came from :
> 
> Using kmalloc() to allocate 48 x 524288 bytes
> 
> Or using vmalloc()
> 
> Are you telling me HugePages are not in play there ?

Oh that's certainly a difference, as kmalloc() will give you the kernel
mapping which can use 1GB Hugepages. But if you just combine these
kmalloc chunks into vmalloc mapping (IIUC that's what your RFC was
doing?), you lose that benefit AFAIK. On the other hand I recall reading
that AMD Zen will have PTE Coalescing [1] which, if true and I
understand that correctly, would indeed result in better TLB usage with
adjacent page table entries pointing to consecutive pages. But perhaps
the starting pte's position will also have to be aligned to make this
work, dunno.

[1]
http://www.anandtech.com/show/10591/amd-zen-microarchiture-part-2-extracting-instructionlevel-parallelism/6