Re: [PATCH v2 0/9] x86/clear_huge_page: multi-page clearing

From: Raghavendra K T
Date: Tue Sep 05 2023 - 12:33:55 EST

Next message: Vignesh Raman: "Re: [PATCH v2 2/7] drm: ci: Force db410c to host mode"
Previous message: Konstantin Meskhidze: "[PATCH] RDMA/uverbs: Fix typo of sizeof argument"
In reply to: Raghavendra K T: "Re: [PATCH v2 0/9] x86/clear_huge_page: multi-page clearing"
Next in thread: Ankur Arora: "Re: [PATCH v2 0/9] x86/clear_huge_page: multi-page clearing"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On 8/31/2023 12:19 AM, Ankur Arora wrote:

This series adds a multi-page clearing primitive, clear_pages(),
which enables more effective use of x86 string instructions by
advertising the real region-size to be cleared.

Region-size can be used as a hint by uarchs to optimize the
clearing.

Also add allow_resched() which marks a code-section as allowing
rescheduling in the irqentry_exit path. This allows clear_pages()
to get by without having to call cond_sched() periodically.
(preempt_model_full() already handles this via
irqentry_exit_cond_resched(), so we handle this similarly for
preempt_model_none() and preempt_model_voluntary().)

Hello Ankur,
Thansk for the patches.

I tried the patches, Improvements look similar to V1 (even without
circuitous chunk optimizations.)
STill we see similar 50-60% improvement for 1G and 2M page sizes.

SUT: Bergamo
CPU family: 25
Model: 160
Thread(s) per core: 2
Core(s) per socket: 128
Socket(s): 2

NUMA:
NUMA node(s): 2
NUMA node0 CPU(s): 0-127,256-383
NUMA node1 CPU(s): 128-255,384-511

Test: Use mmap(MAP_HUGETLB) to demand a fault on 64GB region (NUMA node0), for both base-hugepage-size=2M and 1GB
Current result is with thp = always, but madv also did not make much difference.

perf stat -r 10 -d -d numactl -m 0 -N 0 <test>

time in seconds elapsed (average of 10 runs) (lower = better)

Result:
base: mm/clear_huge_page
patched: x86/clear_huge_page

page-size base patched Improvement %
2M 5.0779 2.50623 50.64
1G 2.50623 1.012439 59.60

More details:

Performance counter stats for 'mm/map_hugetlb' (10 runs):

5,058.71 msec task-clock # 0.996 CPUs utilized ( +- 0.26% )
8 context-switches # 1.576 /sec ( +- 7.23% )
0 cpu-migrations # 0.000 /sec
32,917 page-faults # 6.484 K/sec ( +- 0.00% )
15,797,804,067 cycles # 3.112 GHz ( +- 0.26% ) (35.70%)
2,073,754 stalled-cycles-frontend # 0.01% frontend cycles idle ( +- 1.25% ) (35.71%)
27,508,977 stalled-cycles-backend # 0.17% backend cycles idle ( +- 9.48% ) (35.74%)
1,143,710,651 instructions # 0.07 insn per cycle
# 0.03 stalled cycles per insn ( +- 0.15% ) (35.76%)
243,817,330 branches # 48.028 M/sec ( +- 0.12% ) (35.78%)
357,760 branch-misses # 0.15% of all branches ( +- 1.52% ) (35.75%)
2,540,733,497 L1-dcache-loads # 500.483 M/sec ( +- 0.04% ) (35.74%)
1,093,660,557 L1-dcache-load-misses # 42.98% of all L1-dcache accesses ( +- 0.03% ) (35.71%)
73,335,478 L1-icache-loads # 14.446 M/sec ( +- 0.08% ) (35.70%)
878,378 L1-icache-load-misses # 1.19% of all L1-icache accesses ( +- 2.65% ) (35.68%)
1,025,714 dTLB-loads # 202.049 K/sec ( +- 2.70% ) (35.69%)
405,407 dTLB-load-misses # 37.35% of all dTLB cache accesses ( +- 1.59% ) (35.68%)
2 iTLB-loads # 0.394 /sec ( +- 41.63% ) (35.68%)
40,356 iTLB-load-misses # 1552153.85% of all iTLB cache accesses ( +- 7.18% ) (35.68%)

5.0779 +- 0.0132 seconds time elapsed ( +- 0.26% )

Performance counter stats for 'numactl -m 0 -N 0 x86/map_hugetlb' (10 runs):

2,538.40 msec task-clock # 1.013 CPUs utilized ( +- 0.27% )
4 context-switches # 1.597 /sec ( +- 6.51% )
1 cpu-migrations # 0.399 /sec
32,916 page-faults # 13.140 K/sec ( +- 0.00% )
7,901,830,782 cycles # 3.154 GHz ( +- 0.27% ) (35.67%)
6,590,473 stalled-cycles-frontend # 0.08% frontend cycles idle ( +- 10.31% ) (35.71%)
329,970,288 stalled-cycles-backend # 4.23% backend cycles idle ( +- 13.65% ) (35.74%)
725,811,962 instructions # 0.09 insn per cycle
# 0.80 stalled cycles per insn ( +- 0.37% ) (35.78%)
132,182,704 branches # 52.767 M/sec ( +- 0.26% ) (35.82%)
254,163 branch-misses # 0.19% of all branches ( +- 2.47% ) (35.81%)
2,382,927,453 L1-dcache-loads # 951.262 M/sec ( +- 0.04% ) (35.77%)
1,082,022,067 L1-dcache-load-misses # 45.41% of all L1-dcache accesses ( +- 0.02% ) (35.74%)
47,164,491 L1-icache-loads # 18.828 M/sec ( +- 0.37% ) (35.70%)
474,535 L1-icache-load-misses # 0.99% of all L1-icache accesses ( +- 2.93% ) (35.66%)
1,477,334 dTLB-loads # 589.750 K/sec ( +- 5.12% ) (35.65%)
624,125 dTLB-load-misses # 56.24% of all dTLB cache accesses ( +- 5.66% ) (35.65%)
0 iTLB-loads # 0.000 /sec (35.65%)
1,626 iTLB-load-misses # 7069.57% of all iTLB cache accesses ( +-283.51% ) (35.65%)

2.50623 +- 0.00691 seconds time elapsed ( +- 0.28% )

Performance counter stats for 'numactl -m 0 -N 0 mm/map_hugetlb_1G' (10 runs):

2,506.50 msec task-clock # 0.995 CPUs utilized ( +- 0.17% )
4 context-switches # 1.589 /sec ( +- 9.28% )
0 cpu-migrations # 0.000 /sec
214 page-faults # 84.997 /sec ( +- 0.13% )
7,821,519,053 cycles # 3.107 GHz ( +- 0.17% ) (35.72%)
2,037,744 stalled-cycles-frontend # 0.03% frontend cycles idle ( +- 25.62% ) (35.73%)
6,578,899 stalled-cycles-backend # 0.08% backend cycles idle ( +- 2.65% ) (35.73%)
468,648,780 instructions # 0.06 insn per cycle
# 0.01 stalled cycles per insn ( +- 0.10% ) (35.73%)
116,267,370 branches # 46.179 M/sec ( +- 0.08% ) (35.73%)
111,966 branch-misses # 0.10% of all branches ( +- 2.98% ) (35.72%)
2,294,727,165 L1-dcache-loads # 911.424 M/sec ( +- 0.02% ) (35.71%)
1,076,156,463 L1-dcache-load-misses # 46.88% of all L1-dcache accesses ( +- 0.01% ) (35.70%)
26,093,151 L1-icache-loads # 10.364 M/sec ( +- 0.21% ) (35.71%)
132,944 L1-icache-load-misses # 0.51% of all L1-icache accesses ( +- 0.55% ) (35.70%)
30,925 dTLB-loads # 12.283 K/sec ( +- 5.70% ) (35.71%)
27,437 dTLB-load-misses # 86.22% of all dTLB cache accesses ( +- 1.98% ) (35.70%)
0 iTLB-loads # 0.000 /sec (35.71%)
11 iTLB-load-misses # 62.50% of all iTLB cache accesses ( +-140.21% ) (35.70%)

2.51890 +- 0.00433 seconds time elapsed ( +- 0.17% )

Performance counter stats for 'numactl -m 0 -N 0 x86/map_hugetlb_1G' (10 runs):

1,013.59 msec task-clock # 1.001 CPUs utilized ( +- 0.07% )
2 context-switches # 1.978 /sec ( +- 12.91% )
1 cpu-migrations # 0.989 /sec
213 page-faults # 210.634 /sec ( +- 0.17% )
3,169,391,694 cycles # 3.134 GHz ( +- 0.07% ) (35.53%)
109,925 stalled-cycles-frontend # 0.00% frontend cycles idle ( +- 5.56% ) (35.63%)
950,638,913 stalled-cycles-backend # 30.06% backend cycles idle ( +- 5.06% ) (35.73%)
51,189,571 instructions # 0.02 insn per cycle
# 21.03 stalled cycles per insn ( +- 1.22% ) (35.82%)
9,545,941 branches # 9.440 M/sec ( +- 1.50% ) (35.92%)
86,836 branch-misses # 0.88% of all branches ( +- 3.74% ) (36.00%)
46,109,587 L1-dcache-loads # 45.597 M/sec ( +- 3.92% ) (35.96%)
13,796,172 L1-dcache-load-misses # 41.77% of all L1-dcache accesses ( +- 4.81% ) (35.85%)
1,179,166 L1-icache-loads # 1.166 M/sec ( +- 1.22% ) (35.77%)
21,528 L1-icache-load-misses # 1.90% of all L1-icache accesses ( +- 1.85% ) (35.66%)
14,529 dTLB-loads # 14.368 K/sec ( +- 4.65% ) (35.57%)
8,505 dTLB-load-misses # 67.88% of all dTLB cache accesses ( +- 5.61% ) (35.52%)
0 iTLB-loads # 0.000 /sec (35.52%)
8 iTLB-load-misses # 0.00% of all iTLB cache accesses ( +-267.99% ) (35.52%)

1.012439 +- 0.000723 seconds time elapsed ( +- 0.07% )

Please feel free to carry:

Tested-by: Raghavendra K T <raghavendra.kt@xxxxxxx>
for any minor changes.

Thanks and Regards
- Raghu

Next message: Vignesh Raman: "Re: [PATCH v2 2/7] drm: ci: Force db410c to host mode"
Previous message: Konstantin Meskhidze: "[PATCH] RDMA/uverbs: Fix typo of sizeof argument"
In reply to: Raghavendra K T: "Re: [PATCH v2 0/9] x86/clear_huge_page: multi-page clearing"
Next in thread: Ankur Arora: "Re: [PATCH v2 0/9] x86/clear_huge_page: multi-page clearing"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]