[PATCH v2 04/20] compiler_types.h: Don't instrument __always_inline functions for coverage

[Sasha Levin]

noinstr code may only call other noinstr functions and __always_inline
functions. gcov and Clang's source-based code coverage insert their
instrumentation before inlining, so the instrumentation of an
__always_inline function ends up in every noinstr function it gets
inlined into. With Clang's source-based coverage on x86, 2500 of the
3449 relocations in .noinstr.text and 333 of the 451 in .cpuidle.text
of vmlinux.o point to coverage counters or MC/DC bitmaps. GCC's gcov
does the same. Since these are plain memory updates rather than calls,
objtool's noinstr validation doesn't notice them.

KCOV doesn't have this problem, as it instruments the code after
inlining and checks the attributes of the function that the code ends
up in.

Make __always_inline imply __no_profile when a code coverage
measurement implementation is enabled, so that no noinstr code carries
coverage instrumentation. The cost is that __always_inline functions
don't show up in the coverage data at all, including at their callers
outside of noinstr code. For an x86_64 configuration built with Clang 22
and MC/DC, that removes 556 of 60788 functions (0.9%), 29261 of 1598305
regions (1.8%), 11067 of 591108 branches (1.9%) and 445 of 36561 MC/DC
conditions (1.2%) from the coverage data.

This works around a limitation shared by both compilers: they lower
the coverage instrumentation before inlining, and let __always_inline
override a no_profile_instrument_function mismatch, so they can't drop
the instrumentation that ends up in a noinstr function. Once they can,
this can be limited to compilers that don't.

Signed-off-by: Sasha Levin <sashal@xxxxxxxxxx>
---
 include/linux/compiler_types.h | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
index c5921f1390079..5c8fff84004b4 100644
--- a/include/linux/compiler_types.h
+++ b/include/linux/compiler_types.h
@@ -246,6 +246,17 @@ struct ftrace_likely_data {
  */
 #define __inline__ inline
 
+/*
+ * noinstr code may only call noinstr and __always_inline functions. Code
+ * coverage instrumentation is added before inlining, so the counters of an
+ * __always_inline function would end up in the noinstr code it gets inlined
+ * into: don't instrument __always_inline functions.
+ */
+#ifdef CONFIG_COVERAGE_KERNEL
+#undef __always_inline
+#define __always_inline inline __attribute__((__always_inline__)) __no_profile
+#endif
+
 /*
  * GCC does not warn about unused static inline functions for -Wunused-function.
  * Suppress the warning in clang as well by using __maybe_unused, but enable it
-- 
2.53.0