Re: arm64 + ARM64_64K_PAGES=y

[Grygorii Strashko]

On 11/12/18 8:27 AM, Andre Przywara wrote:
> On Fri, 9 Nov 2018 13:15:47 -0600
> Grygorii Strashko <grygorii.strashko@xxxxxx> wrote:
> 
> Hi,
> 
>> On 11/8/18 12:14 PM, Grygorii Strashko wrote:
>>>
>>>
>>> On 11/8/18 6:00 AM, Sebastian Andrzej Siewior wrote:
>>>> On 2018-11-06 15:34:55 [-0600], Grygorii Strashko wrote:
>>>>> Hi All,
>>>> Hi,
>>>>   
>>>>> Do anybody tried to use ARM64 RT with 76K pages enabled?
>>>>
>>>> 75 would be an off by one but this :)
>>>
>>> Ops 8-). at least subj is correct.
>>>    
>>>>   
>>>>> My attempt shows that enabling CONFIG_ARM64_64K_PAGES=y
>>>>> increases latencies by ~30%
> 
> That's not really surprising. Performance on systems using a bigger page
> size granules might have some trade-offs (bigger memory overhead, worse
> cache utilization), so 64K pages might not be really great for your
> particular workload. You would probably need a real performance
> analysis (using perf, for instance) to pinpoint TLB misses as your
> bottleneck.
> 
>>>>> cyclictest -n -m -Sp98 -q -D2m with  =y
>>>>>
>>>>>
>>>>> T: 0 (Â 772) P:98 I:1000 C: 120000 Min:ÂÂÂÂÂ 7 Act:ÂÂ 13 Avg:
>>>>> 10 Max:ÂÂÂÂÂ 85 T: 1 (Â 773) P:98 I:1500 C:Â 79998 Min:ÂÂÂÂÂ 7
>>>>> Act:ÂÂ 13 Avg:ÂÂ 10 Max:ÂÂÂÂÂ 71 T: 2 (Â 774) P:98 I:2000 C:
>>>>> 59997 Min:ÂÂÂÂÂ 7 Act:ÂÂ 11 Avg:ÂÂ 11 Max:ÂÂÂÂÂ 64 T: 3 (Â 775)
>>>>> P:98 I:2500 C:Â 47996 Min:ÂÂÂÂÂ 7 Act:ÂÂ 14 Avg:ÂÂ 12 Max:ÂÂÂÂÂ 66
>>>>>
>>>>>
>>>>> cyclictest -n -m -Sp98 -q -D2m with CONFIG_ARM64_64K_PAGES=n
>>>>>
>>>>>
>>>>> T: 0 (Â 697) P:98 I:1000 C: 120000 Min:ÂÂÂÂÂ 7 Act:ÂÂ 10 Avg:
>>>>> 9 Max:ÂÂÂÂÂ 38 T: 1 (Â 698) P:98 I:1500 C:Â 79987 Min:ÂÂÂÂÂ 7
>>>>> Act:ÂÂ 10 Avg:ÂÂ 10 Max:ÂÂÂÂÂ 32 T: 2 (Â 699) P:98 I:2000 C:
>>>>> 59981 Min:ÂÂÂÂÂ 7 Act:ÂÂ 14 Avg:ÂÂ 11 Max:ÂÂÂÂÂ 46 T: 3 (Â 700)
>>>>> P:98 I:2500 C:Â 47977 Min:ÂÂÂÂÂ 6 Act:ÂÂ 11 Avg:ÂÂ 10 Max:
>>>>> 45
>>>>
>>>> So this is an idle system?
>>>
>>> Yes (in general) - it's collected with systemd, so some daemons are
>>> active.
>>>> The Kconfig help says "faster TLB lookup". Interesting.
>>>> Are the 16k pages in between (latency wise) by any chance?
>>>
>>> I'll try it.
>>
>> no i'll not, at least not fast. with 16k pages enabled I can't boot
>> TI 4.14 kernel
>> -  4.14.71-rt44.
>> No msg in log, just "Starting kernel ..."
> 
> You need a core that actually supports 16K pages (supporting
> certain page size granules is architecturally optional).
>  From the Arm Ltd. cores it's Cortex-A73, A75 or A55, possibly other
> newer ones as well. Cortex-A53, A57 and A72 do not support 16k pages.

Thank a lot for you reply.

-- 
regards,
-grygorii