Re: [PATCH v8 00/10] Multi-size THP for anonymous memory
From: Ryan Roberts
Date: Wed Dec 06 2023 - 05:08:09 EST
On 05/12/2023 14:19, Kefeng Wang wrote:
>
>
> On 2023/12/4 18:20, Ryan Roberts wrote:
>> Hi All,
>>
>> A new week, a new version, a new name... This is v8 of a series to implement
>> multi-size THP (mTHP) for anonymous memory (previously called "small-sized THP"
>> and "large anonymous folios"). Matthew objected to "small huge" so hopefully
>> this fares better.
>>
>> The objective of this is to improve performance by allocating larger chunks of
>> memory during anonymous page faults:
>>
>> 1) Since SW (the kernel) is dealing with larger chunks of memory than base
>> pages, there are efficiency savings to be had; fewer page faults, batched PTE
>> and RMAP manipulation, reduced lru list, etc. In short, we reduce kernel
>> overhead. This should benefit all architectures.
>> 2) Since we are now mapping physically contiguous chunks of memory, we can take
>> advantage of HW TLB compression techniques. A reduction in TLB pressure
>> speeds up kernel and user space. arm64 systems have 2 mechanisms to coalesce
>> TLB entries; "the contiguous bit" (architectural) and HPA (uarch).
>>
>> This version changes the name and tidies up some of the kernel code and test
>> code, based on feedback against v7 (see change log for details).
>>
>> By default, the existing behaviour (and performance) is maintained. The user
>> must explicitly enable multi-size THP to see the performance benefit. This is
>> done via a new sysfs interface (as recommended by David Hildenbrand - thanks to
>> David for the suggestion)! This interface is inspired by the existing
>> per-hugepage-size sysfs interface used by hugetlb, provides full backwards
>> compatibility with the existing PMD-size THP interface, and provides a base for
>> future extensibility. See [8] for detailed discussion of the interface.
>>
>> This series is based on mm-unstable (715b67adf4c8).
>>
>>
>> Prerequisites
>> =============
>>
>> Some work items identified as being prerequisites are listed on page 3 at [9].
>> The summary is:
>>
>> | item | status |
>> |:------------------------------|:------------------------|
>> | mlock | In mainline (v6.7) |
>> | madvise | In mainline (v6.6) |
>> | compaction | v1 posted [10] |
>> | numa balancing | Investigated: see below |
>> | user-triggered page migration | In mainline (v6.7) |
>> | khugepaged collapse | In mainline (NOP) |
>>
>> On NUMA balancing, which currently ignores any PTE-mapped THPs it encounters,
>> John Hubbard has investigated this and concluded that it is A) not clear at the
>> moment what a better policy might be for PTE-mapped THP and B) questions whether
>> this should really be considered a prerequisite given no regression is caused
>> for the default "multi-size THP disabled" case, and there is no correctness
>> issue when it is enabled - its just a potential for non-optimal performance.
>>
>> If there are no disagreements about removing numa balancing from the list (none
>> were raised when I first posted this comment against v7), then that just leaves
>> compaction which is in review on list at the moment.
>>
>> I really would like to get this series (and its remaining comapction
>> prerequisite) in for v6.8. I accept that it may be a bit optimistic at this
>> point, but lets see where we get to with review?
>>
>>
>> Testing
>> =======
>>
>> The series includes patches for mm selftests to enlighten the cow and khugepaged
>> tests to explicitly test with multi-size THP, in the same way that PMD-sized
>> THP is tested. The new tests all pass, and no regressions are observed in the mm
>> selftest suite. I've also run my usual kernel compilation and java script
>> benchmarks without any issues.
>>
>> Refer to my performance numbers posted with v6 [6]. (These are for multi-size
>> THP only - they do not include the arm64 contpte follow-on series).
>>
>> John Hubbard at Nvidia has indicated dramatic 10x performance improvements for
>> some workloads at [11]. (Observed using v6 of this series as well as the arm64
>> contpte series).
>>
>> Kefeng Wang at Huawei has also indicated he sees improvements at [12] although
>> there are some latency regressions also.
>
> Hi Ryan,
>
> Here is some test results based on v6.7-rc1 +
> [PATCH v7 00/10] Small-sized THP for anonymous memory +
> [PATCH v2 00/14] Transparent Contiguous PTEs for User Mappings
>
> case1: basepage 64K
> case2: basepage 4K + thp=64k + PAGE_ALLOC_COSTLY_ORDER = 3
> case3: basepage 4K + thp=64k + PAGE_ALLOC_COSTLY_ORDER = 4
Thanks for sharing these results. With the exception of a few outliers, It looks
like the ~rough conclusion is that bandwidth improves, but not as much as 64K
base pages, and latency regresses, but also not as much as 64K base pages?
I expect that over time, as we add more optimizations, we will get bandwidth
closer to 64K base pages; one crucial one is getting executable file-backed
memory into contpte mappings, for example.
It's probably not time to switch PAGE_ALLOC_COSTLY_ORDER quite yet; but
something to keep an eye on and consider down the road?
Thanks,
Ryan
>
> The results is compared with basepage 4K on Kunpeng920.
>
> Note,
> - The test based on ext4 filesystem and THP=2M is disabled.
> - The results were not analyzed, it is for reference only,
> as some values of test items are not consistent.
>
> 1) Unixbench 1core
> Index_Values_1core case1 case2 case3
> Dhrystone_2_using_register_variables 0.28% 0.39% 0.17%
> Double-Precision_Whetstone -0.01% 0.00% 0.00%
> Execl_Throughput *21.13%* 2.16% 3.01%
> File_Copy_1024_bufsize_2000_maxblocks -0.51% *8.33%* *8.76%*
> File_Copy_256_bufsize_500_maxblocks 0.78% *11.89%* *10.85%*
> File_Copy_4096_bufsize_8000_maxblocks 7.42% 7.27% *10.66%*
> Pipe_Throughput -0.24% *6.82%* *5.08%*
> Pipe-based_Context_Switching 1.38% *13.49%* *9.91%*
> Process_Creation *32.46%* 4.30% *8.54%*
> Shell_Scripts_(1_concurrent) *31.67%* 1.92% 2.60%
> Shell_Scripts_(8_concurrent) *40.59%* 1.30% *5.29%*
> System_Call_Overhead 3.92% *8.13% 2.96%
>
> System_Benchmarks_Index_Score 10.66% 5.39% 5.58%
>
> For 1core,
> - case1 wins on Execl_Throughput/Process_Creation/Shell_Scripts
> a lot, and score higher 10.66% vs basepage 4K.
> - case2/3 wins on File_Copy/Pipe and score higher 5%+ than basepage 4K,
> also case3 looks better on Shell_Scripts_(8_concurrent) than case2.
>
> 2) Unixbench 128core
> Index_Values_128core case1 case2 case3
> Dhrystone_2_using_register_variables 2.07% -0.03% -0.11%
> Double-Precision_Whetstone -0.03% 0.00% 0.00%
> Execl_Throughput *39.28%* -4.23% 1.93%
> File_Copy_1024_bufsize_2000_maxblocks 5.46% 1.30% 4.20%
> File_Copy_256_bufsize_500_maxblocks -8.89% *6.56% *5.02%*
> File_Copy_4096_bufsize_8000_maxblocks 3.43% *-5.46%* 0.56%
> Pipe_Throughput 3.80% *7.69% *7.80%*
> Pipe-based_Context_Switching *7.62%* 0.95% 4.69%
> Process_Creation *28.11%* -2.79% 2.40%
> Shell_Scripts_(1_concurrent) *39.68%* 1.86% *5.30%*
> Shell_Scripts_(8_concurrent) *41.35%* 2.49% *7.16%*
> System_Call_Overhead -1.55% -0.04% *8.23%*
>
> System_Benchmarks_Index_Score 12.08% 0.63% 3.88%
>
> For 128core,
> - case1 wins on Execl_Throughput/Process_Creation/Shell_Scripts
> a lot, also good at Pipe-based_Context_Switching, and score higher
> 12.08% vs basepage 4K.
> - case2/case3 wins on File_Copy_256/Pipe_Throughput, but case2 is
> not better than basepage 4K, case3 wins 3.88%.
>
> 3) Lmbench Processor_processes
> Processor_Processes case1 case2 case3
> null_call 1.76% 0.40% 0.65%
> null_io -0.76% -0.38% -0.23%
> stat *-16.09%* *-12.49%* 4.22%
> open_close -2.69% 4.51% 3.21%
> slct_TCP -0.56% 0.00% -0.44%
> sig_inst -1.54% 0.73% 0.70%
> sig_hndl -2.85% 0.01% 1.85%
> fork_proc *23.31%* 8.77% -5.42%
> exec_proc *13.22%* -0.30% 1.09%
> sh_proc *14.04%* -0.10% 1.09%
>
> - case1 is much better than basepage 4K, same as Unixbench test,
> case2 is better on fork_proc, but case3 is worse
> - note: the variance of fork/exec/sh is bigger than others
>
> 4) Lmbench Context_switching_ctxsw
> Context_switching_ctxsw case1 case2 case3
> 2p/0K -12.16% -5.29% -1.86%
> 2p/16K -11.26% -3.71% -4.53%
> 2p/64K -2.60% 3.84% -1.98%
> 8p/16K -7.56% -1.21% -0.88%
> 8p/64K 5.10% 4.88% 1.19%
> 16p/16K -5.81% -2.44% -3.84%
> 16p/64K 4.29% -1.94% -2.50%
> - case1/2/3 worse than basepage 4K and case1 is the worst.
>
> 4) Lmbench Local_latencies
> Local_latencies case1 case2 case3
> Pipe -9.23% 0.58% -4.34%
> AF_UNIX -5.34% -1.76% 3.03%
> UDP -6.70% -5.96% -9.81%
> TCP -7.95% -7.58% -5.63%
> TCP_conn -213.99% -227.78% -659.67%
> - TCP_conn is very unreliable, ignore it
> - case1/2/3 slower than basepage 4K
>
> 5) Lmbench File_&_VM_latencies
> File_&_VM_latencies case1 case2 case3
> 10K_File_Create 2.60% -0.52% 2.66%
> 10K_File_Delete -2.91% -5.20% -2.11%
> 10K_File_Create 10.23% 1.18% 0.12%
> 10K_File_Delete -17.76% -2.97% -1.49%
> Mmap_Latency *63.05%* 2.57% -0.96%
> Prot_Fault 10.41% -3.21% *-19.11%*
> Page_Fault *-132.01%* 2.35% -0.79%
> 100fd_selct -1.20% 0.10% 0.31%
> - case1 is very good at Mmap_Latency and not good at Page_fault
> - case2/3 slower on Prot_Faul/10K_FILE_Delete vs basepage 4k,
> the rest doesn't look much different.
>
> 6) Lmbench Local_bandwidths
> Local_bandwidths case1 case2 case3
> Pipe 265.22% 15.44% 11.33%
> AF_UNIX 13.41% -2.66% 2.63%
> TCP -1.30% 25.90% 2.48%
> File_reread 14.79% 31.52% -14.16%
> Mmap_reread 27.47% 49.00% -0.11%
> Bcopy(libc) 2.58% 2.45% 2.46%
> Bcopy(hand) 25.78% 22.56% 22.68%
> Mem_read 38.26% 36.80% 36.49%
> Mem_write 10.93% 3.44% 3.12%
>
> - case1 is very good at bandwidth, case2 is better than basepage 4k
> but lower than case1, case3 is bad at File_reread
>
> 7)Lmbench Memory_latencies
> Memory_latencies case1 case2 case3
> L1_$ 0.02% 0.00% -0.03%
> L2_$ -1.56% -2.65% -1.25%
> Main_mem 50.82% 32.51% 33.47%
> Rand_mem 15.29% -8.79% -8.80%
>
> - case1 also good at Main/Rand mem access latencies,
> - case2/case3 is better at Main_mem, but worse at Rand_mem.
>
> Tested-by: Kefeng Wang <wangkefeng.wang@xxxxxxxxxx>
>
>
>
>
>
>
>