[RFC PATCH 04/20] compiler_types.h: Don't instrument __always_inline functions for coverage

[Sasha Levin]

noinstr code may only call other noinstr functions and __always_inline
functions. gcov and Clang's source-based code coverage insert their
instrumentation before inlining, so the instrumentation of an
__always_inline function ends up in every noinstr function it gets
inlined into. With Clang's source-based coverage on x86, 2500 of the
3449 relocations in .noinstr.text and 333 of the 451 in .cpuidle.text
of vmlinux.o point to coverage counters or MC/DC bitmaps. GCC's gcov
does the same. Since these are plain memory updates rather than calls,
objtool's noinstr validation doesn't notice them.

KCOV doesn't have this problem, as it instruments the code after
inlining and checks the attributes of the function that the code ends
up in.

Make __always_inline imply __no_profile when a code coverage
measurement implementation is enabled, so that no noinstr code carries
coverage instrumentation. The cost is that __always_inline functions
don't show up in the coverage data at all, including at their callers
outside of noinstr code. For an x86_64 configuration built with Clang 22
and MC/DC, that removes 556 of 60788 functions (0.9%), 29261 of 1598305
regions (1.8%), 11067 of 591108 branches (1.9%) and 445 of 36561 MC/DC
conditions (1.2%) from the coverage data.

Leave unoptimized objects alone: GCC refuses to inline an
__always_inline function across a no_profile_instrument_function
mismatch at -O0, and those objects don't contain noinstr code.

Signed-off-by: Sasha Levin <sashal@xxxxxxxxxx>
---
 include/linux/compiler_types.h | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
index c5921f1390079..c3298d29a9425 100644
--- a/include/linux/compiler_types.h
+++ b/include/linux/compiler_types.h
@@ -246,6 +246,19 @@ struct ftrace_likely_data {
  */
 #define __inline__ inline
 
+/*
+ * noinstr code may only call noinstr and __always_inline functions. Code
+ * coverage instrumentation is added before inlining, so the counters of an
+ * __always_inline function would end up in the noinstr code it gets inlined
+ * into: don't instrument __always_inline functions. GCC refuses to inline
+ * across a no_profile_instrument_function mismatch without optimization, so
+ * leave the rare unoptimized objects alone; they don't contain noinstr code.
+ */
+#if defined(CONFIG_COVERAGE_KERNEL) && defined(__OPTIMIZE__)
+#undef __always_inline
+#define __always_inline inline __attribute__((__always_inline__)) __no_profile
+#endif
+
 /*
  * GCC does not warn about unused static inline functions for -Wunused-function.
  * Suppress the warning in clang as well by using __maybe_unused, but enable it
-- 
2.53.0