Re: [PATCH v4] pgo: add clang's Profile Guided Optimization infrastructure

[Nathan Chancellor]

Hi Bill,

On Tue, Jan 12, 2021 at 10:19:58PM -0800, Bill Wendling wrote:
> From: Sami Tolvanen <samitolvanen@xxxxxxxxxx>
> 
> Enable the use of clang's Profile-Guided Optimization[1]. To generate a
> profile, the kernel is instrumented with PGO counters, a representative
> workload is run, and the raw profile data is collected from
> /sys/kernel/debug/pgo/profraw.
> 
> The raw profile data must be processed by clang's "llvm-profdata" tool
> before it can be used during recompilation:
> 
>   $ cp /sys/kernel/debug/pgo/profraw vmlinux.profraw
>   $ llvm-profdata merge --output=vmlinux.profdata vmlinux.profraw
> 
> Multiple raw profiles may be merged during this step.
> 
> The data can now be used by the compiler:
> 
>   $ make LLVM=1 KCFLAGS=-fprofile-use=vmlinux.profdata ...
> 
> This initial submission is restricted to x86, as that's the platform we
> know works. This restriction can be lifted once other platforms have
> been verified to work with PGO.
> 
> Note that this method of profiling the kernel is clang-native, unlike
> the clang support in kernel/gcov.
> 
> [1] https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization
> 
> Signed-off-by: Sami Tolvanen <samitolvanen@xxxxxxxxxx>
> Co-developed-by: Bill Wendling <morbo@xxxxxxxxxx>
> Signed-off-by: Bill Wendling <morbo@xxxxxxxxxx>
> Change-Id: Ic78e69c682286d3a44c4549a0138578c98138b77

Small nit: This should be removed.

I applied this patch on top of v5.11-rc3, built it with LLVM 12
(f1d5cbbdee5526bc86eac0a5652b115d9bc158e5 + D94470) with Microsoft's
WSL 5.4 config [1] + CONFIG_PGO_CLANG=y, and ran it on WSL2.

$ zgrep PGO /proc/config.gz
# Profile Guided Optimization (PGO) (EXPERIMENTAL)
CONFIG_ARCH_SUPPORTS_PGO_CLANG=y
CONFIG_PGO_CLANG=y
# end of Profile Guided Optimization (PGO) (EXPERIMENTAL)

However, I see an issue with actually using the data:

$ sudo -s
# mount -t debugfs none /sys/kernel/debug
# cp -a /sys/kernel/debug/pgo/profraw vmlinux.profraw
# chown nathan:nathan vmlinux.profraw
# exit
$ tc-build/build/llvm/stage1/bin/llvm-profdata merge --output=vmlinux.profdata vmlinux.profraw
warning: vmlinux.profraw: Invalid instrumentation profile data (bad magic)
error: No profiles could be merged.

Am I holding it wrong? :) Note, this is virtualized, I do not have any
"real" x86 hardware that I can afford to test on right now.

[1]: https://github.com/microsoft/WSL2-Linux-Kernel/raw/linux-msft-wsl-5.4.y/Microsoft/config-wsl

Cheers,
Nathan